Commit Graph

469 Commits (e09ade34cdef1c02316a63dfb0c8b04d79f931ed)

Author SHA1 Message Date
David Yip 976347fa78 Merge remote-tracking branch 'origin/master' into merge-upstream 2018-05-04 03:23:57 -05:00
David Yip c87f1d99e4 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	.travis.yml
 	Gemfile.lock
 	README.md
 	app/controllers/settings/follower_domains_controller.rb
 	app/controllers/statuses_controller.rb
 	app/javascript/mastodon/locales/ja.json
 	app/lib/feed_manager.rb
 	app/models/media_attachment.rb
 	app/models/mute.rb
 	app/models/status.rb
 	app/services/mute_service.rb
 	app/views/home/index.html.haml
 	app/views/stream_entries/_simple_status.html.haml
 	config/locales/ca.yml
 	config/locales/en.yml
 	config/locales/es.yml
 	config/locales/fr.yml
 	config/locales/nl.yml
 	config/locales/pl.yml
 	config/locales/pt-BR.yml
 	config/themes.yml
2018-05-03 17:23:44 -05:00
ThibG f2ecedaab5 Fixes/do not override timestamps (#7336)
* Revert "Fixes/do not override timestamps (#7331)"

This reverts commit 9c1e1005f4.

* Document Snowflake ID corner-case a bit more

Snowflake IDs are used for two purposes: making object identifiers harder to
guess and ensuring they are in chronological order. For this reason, they
are based on the `created_at` attribute of the object.

Unfortunately, inserting items with older snowflakes IDs will break the
assumption of consumers of the paging APIs that new items will always have
a greater identifier than the last seen one.

* Add `override_timestamps` virtual attribute to not correlate snowflake ID with created_at
2018-05-03 23:02:46 +02:00
ThibG 9c1e1005f4 Fixes/do not override timestamps (#7331)
* Do not override timestamps for incoming toots

* Remove every reference to override_timestamps

Statuses are now created with the announced publishing date
and are only pushed to timelines if that date is at most
6 hours earlier than the time at which it is processed.
2018-05-03 13:33:08 +02:00
Eugen Rochko 92c5474861 Fix n+1 queries in StatusThreadingConcern (#7321) 2018-05-03 10:41:58 +02:00
Eugen Rochko ca1c696dbd Slightly reduce RAM usage (#7301)
* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts
2018-05-02 18:58:48 +02:00
ThibG 616cfab97c Accept actor object updates from all supported actor types (#7312) 2018-05-02 16:08:16 +02:00
Eugen Rochko c5fa8041c2 Remove most behaviour disparities between blocks and mutes (#7231)
* Remove most behaviour disparities between blocks and mutes

The only differences between block and mute should be:

- Mutes can optionally NOT affect notifications
- Mutes should not be visible to the muted

Fix #7230
Fix #5713

* Do not allow boosting someone you blocked

Fix #7248

* Do not allow favouriting someone you blocked

* Fix nil error in StatusPolicy
2018-05-02 15:50:20 +02:00
Eugen Rochko 192746ec3e Guard against nil URLs in Request class (#7284)
Fix #7265
2018-05-02 15:44:22 +02:00
Surinna Curtis 72793b3fc7 Support Actors/Statuses with multiple types (#7305)
* Add equals_or_includes_any? helper in JsonLdHelper

* Support arrays in JSON-LD type fields for actors/tags/objects.

* Spec for resolving accounts with extension types

* Style tweaks for codeclimate
2018-05-02 12:40:24 +02:00
Eugen Rochko 62a50fcc75 Fix missing updated_at attribute on emoji EntityCache (#7297)
Just don't try to save space by only selecting few attributes. If
anyone is wondering, this is needed because the emoji entity cache
is not really only used for entities, it's accessed again to
generate Emoji tags in ActivityPub/OStatus, so a lot more properties
are used than what is needed in HTML alone...
2018-04-30 22:49:33 +02:00
Eugen Rochko aa54b99f3f Fix missing domain attribute in EntityCache for emoji (#7290) 2018-04-30 09:12:55 +02:00
Eugen Rochko 1a653fff95 Add entity cache (#7271)
* Add entity cache

Use a caching layer for mentions and custom emojis that are
dynamically extracted from text.

Reduce duplicate text extractions

* Fix code style issue
2018-04-27 01:38:10 +02:00
MIYAGI Hikaru 28808f638e HTTP proxy support for outgoing request, manage access to hidden service (#7134)
* Add support for HTTP client proxy

* Add access control for darknet

Supress error when access to darknet via transparent proxy

* Fix the codes pointed out

* Lint

* Fix an omission + lint

* any? -> include?

* Change detection method to regexp to avoid test fail
2018-04-25 02:14:49 +02:00
Eugen Rochko 24330e4fa7 Add RSS feeds for end-users (#7259)
* Add RSS feed for accounts

* Add RSS feeds for hashtags

* Fix code style issues

* Fix code style issues
2018-04-25 02:10:02 +02:00
Eugen Rochko 39f36c0901 Detect and prevent image bombs, max. processable dimension 4096^2 (#7229) 2018-04-23 09:16:38 +02:00
Eugen Rochko 599aa39aca Remove "nsfw" category for sensitive statuses in OStatus serializer (#7048)
Fix #7011
2018-04-22 22:09:03 +02:00
David Yip 9697c6d9f0 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	db/schema.rb
2018-04-14 09:12:50 -05:00
Eugen Rochko 466442b764 Add bio fields (#6645)
* Add bio fields

- Fix #3211
- Fix #232
- Fix #121

* Display bio fields in web UI

* Fix output of links and missing fields

* Federate bio fields over ActivityPub as PropertyValue

* Improve how the fields are stored, add to Edit profile form

* Add rel=me to links in fields

Fix #121
2018-04-14 12:41:08 +02:00
David Yip 337fc136fd Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/serializers/initial_state_serializer.rb

The glitch flavour isn't yet pulling custom emoji data on its own (see
https://github.com/tootsuite/mastodon/pull/7047).  Once that gets into
the glitch flavour, we can eliminate the custom_emojis load.
2018-04-08 19:05:02 -05:00
Levi Bard 6828abc1d7 Enable updating additional account information from user preferences via rest api (#6789)
* Enable updating additional account information from user preferences via rest api
Resolves #6553

* Pacify rubocop

* Decoerce incoming settings in UserSettingsDecorator

* Create user preferences hash directly from incoming credentials instead of going through ActionController::Parameters

* Clean up user preferences update

* Use ActiveModel::Type::Boolean instead of manually checking stringified number equivalence
2018-04-08 13:43:10 +02:00
Eugen Rochko 608b4096ea Forward deletes on the same path as reply forwarding (#7058)
* Forward deletes on the same path as reply forwarding

* Remove trailing whitespace
2018-04-07 18:54:46 +02:00
David Yip 21394c5462 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/javascript/mastodon/locales/en.json
 	app/javascript/mastodon/locales/ja.json
 	app/javascript/mastodon/locales/pl.json
 	app/views/accounts/_header.html.haml
2018-04-01 20:24:50 -05:00
David Underwood 44274257de [WIP] Enable custom emoji on account pages and in the sidebar (#6124)
Federate custom emojis with accounts
2018-04-01 23:55:42 +02:00
Jenkins 658440838b Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-30 19:17:17 +00:00
ThibG 1cda8b247a Fix compatibility with PeerTube (#6968)
* Support fetching objects of convertible types by URL (fixes #6924)

* Ignore invalid hashtags
2018-03-30 15:44:54 +02:00
ThibG f7a96189b6 Do not ignore unknown media attachments, only skip them (#6948)
That way, they are displayed in a list below the corresponding toot.
2018-03-29 00:52:24 +02:00
David Yip e6e537d002 Merge pull request #400 from vulpineclub/dev-gs-392-frontmatter
Remove ignored 'u' parameter on regexp (#392)
2018-03-27 13:06:28 -05:00
Rey Tucker b65fc60334 Remove ignored 'u' parameter on regexp (#392) 2018-03-27 11:30:49 -04:00
David Yip 7cdf2b05b6 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/javascript/styles/mastodon/components.scss
 	app/models/media_attachment.rb
2018-03-27 10:26:47 -05:00
Akihiko Odaki acece7a2e6 Validate HTTP response length while receiving (#6891)
to_s method of HTTP::Response keeps blocking while it receives the whole
content, no matter how it is big. This means it may waste time to receive
unacceptably large files. It may also consume memory and disk in the
process. This solves the inefficency by checking response length while
receiving.
2018-03-26 14:02:10 +02:00
Jenkins ae55717f50 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-25 15:17:21 +00:00
Akihiko Odaki 11c19f6cc9 Close http connection in perform method of Request class (#6889)
HTTP connections must be explicitly closed in many cases, and letting
perform method close connections makes its callers less redundant and
prevent them from forgetting to close connections.
2018-03-24 12:49:54 +01:00
Jenkins b1453c0dbf Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-20 19:17:20 +00:00
Rey Tucker 4c91be94d3 request: in the event of failure, try other IPs (#6761) (#6813)
* request: in the event of failure, try other IPs (#6761)

In the case where a name has multiple A/AAAA records, we should
try subsequent records instead of immediately failing when we have a
failure on the first IP address.

This significantly improves delivery success when there are network
connectivity problems affecting only IPv4 or IPv6.

* fix method call style

* request_spec: adjust test case to use Addrinfo

* request: Request/open: move private addr check to within begin/rescue

* request_spec: add case to test failover, fix exception check

* Double Addrinfo.foreach so that it correctly yields instances
2018-03-20 09:06:08 +01:00
Jenkins eba1b109db Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-19 20:17:18 +00:00
ThibG 99790407e2 Serialize mentions in the order they are added (#6836)
Up until now, the order seemed to be in the *opposite* order,
which caused the WebUI to populate mentions in reversed order
when replying to toots local to one's instance.
2018-03-19 20:19:35 +01:00
David Yip b0993a5fdf Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	config/i18n-tasks.yml
2018-03-18 20:21:17 -05:00
Eugen Rochko f0bc8e8a89 Ignore media validation when attaching to status during processing (#6822)
Fix #6821
2018-03-19 01:51:19 +01:00
Jenkins 54608d0486 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-09 00:17:17 +00:00
MitarashiDango 894226c288 fix validation error (media only status) (#6684)
* fix validation error (media only status)

* Incorporating review suggestions

* Reflect similar fix to OStatus side

* Fix not to include media in transaction

* Restore the limit of the number of media

* Fix not to return nil
2018-03-08 01:22:47 +01:00
Eugen Rochko 0c2a08adeb Remove text requirement when media attached from statuses (#6672) 2018-03-07 08:28:52 +01:00
Eugen Rochko 29f412da2c Fix #6657 - Use target instead of origin in Remove activity (#6664) 2018-03-07 03:54:46 +01:00
Eugen Rochko b1f8dfd3cf Federate pinned statuses over ActivityPub (#6610)
* Federate pinned statuses over ActivityPub

* Display pinned toots in web UI

Fix #6117

* Fix migration

* Fix tests

* Update outbox_serializer.rb

* Update remove_serializer.rb

* Update add_serializer.rb

* Update fetch_featured_collection_service.rb
2018-03-04 09:19:11 +01:00
David Yip c08c971dd3 Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
 	README.md
 	app/controllers/follower_accounts_controller.rb
 	app/controllers/following_accounts_controller.rb
 	app/serializers/rest/instance_serializer.rb
 	app/views/stream_entries/_simple_status.html.haml
 	config/locales/simple_form.ja.yml
2018-03-02 21:46:44 -06:00
Eugen Rochko 5f70533e8a Fix #5708: Reject->Follow will remove the follow if it exists (#6571) 2018-02-28 06:55:06 +01:00
Eugen Rochko 0ebd2a1718 Federated reports (#6570)
* Fix #2176: Federated reports

* UI for federated reports

* Add spec for ActivityPub Flag handler

* Add spec for ReportService
2018-02-28 06:54:55 +01:00
Akihiko Odaki a5a434a8f6 Raise Mastodon::HostValidationError when host for HTTP request is private (#6410) 2018-02-24 19:16:11 +01:00
Jenkins b4f099ac0a Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-24 05:17:10 +00:00
Akihiko Odaki 7ac6e09dde Do not push status to feed if its reblog is already inserted (#6488)
A complemental change for precompute_feed_service_spec.rb also fixes its
random failure which is caused by the Snowlake randomization of the order
of an original status and its reblog.
2018-02-24 05:40:18 +01:00
imncls c0aabbec0f Merge branch 'master' of https://github.com/tootsuite/mastodon
# Conflicts:
#	app/controllers/settings/exports_controller.rb
#	app/models/media_attachment.rb
#	app/models/status.rb
#	app/views/about/show.html.haml
#	docker_entrypoint.sh
#	spec/views/about/show.html.haml_spec.rb
2018-02-23 23:28:31 +09:00
Eugen Rochko 865c7e7178 Focal points (#6520)
* Add focus param to media API, center thumbnails on focus point

* Add UI for setting a focal point

* Improve focal point icon on upload item

* Use focal point in upload preview

* Add focalPoint property to ActivityPub

* Don't show focal point button for non-image attachments
2018-02-22 00:35:46 +01:00
Eugen Rochko c34ff9fb82 Fix avatar and header issues by using custom geometry detector (#6515)
* Fix avatar and header issues by using custom geometry detector

Revert a part of #6508. The file passed to dynamic styles method
was not actually a file, but an instance of Paperclip::Attachment,
which broke all styles by always returning {} from the method.

One problem with GIF avatars was that Paperclip::GeometryDetector
reported wrong dimensions for them, e.g. 120x120 GIF avatar would
for some reason be detected as 120x53. By writing our own geometry
parser, we can use FastImage, which also happens to be faster than
ImageMagick, to detect image dimensions, which are also correct.

Unfortunately, this PR does not implement skipping a `convert`
entirely if the dimensions are already correct, as I found no easy
way to write that behaviour into Paperclip without rewriting the
Paperclip::Thumbnail class.

* Only invoke convert if dimension or format needs to be changed
2018-02-21 03:40:12 +01:00
Jenkins e768ef35b1 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-19 15:17:14 +00:00
Eugen Rochko 3f2a113d32 Push discovered status through streaming API within a time window (#6484)
Time window of 6 hours
2018-02-17 14:28:48 +01:00
Jenkins 3a1f58e9eb Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-11 04:10:16 +00:00
David Yip d9cc3b60fb Merge remote-tracking branch 'origin/merge-upstream' 2018-02-10 22:08:45 -06:00
David Yip 00cb763893 Glitch::FilterHelper -> Glitch::KeywordMuteHelper. #234.
The class helps out with keyword mutes, not just some general concept of
"filtering".
2018-02-10 10:36:15 -06:00
David Yip e365675e0a Strip HTML from keyword mute input. #234. 2018-02-10 02:40:27 -06:00
Eugen Rochko 7ca5a06505 Full-text search for authorized statuses (#6423)
* Add full-text search for authorized statuses

- Search API will return statuses that match the query
- Only for logged in users
- Only if you are author of the status,
- Or you were mentioned in it
- Or you favourited or reblogged it
- Configuration over `ES_ENABLED`, `ES_HOST`, `ES_PORT`, `ES_PREFIX`
- Run `rails chewy:deploy` to create & populate index

Fix #5880
Fix #4293
Fix #1152

* Add commented out docker-compose configuration for ES container

* Optimize index import, filter search results

* Add basic normalization to the index

* Add better stemming and normalization to the index

* Skip webfinger request if search query includes both @ and a space

* Fix code style

* Visually separate search result sections

* Fix code style issues
2018-02-09 23:04:47 +01:00
David Yip 799b205766 Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	.travis.yml
	app/lib/user_settings_decorator.rb
	app/models/user.rb
	app/serializers/initial_state_serializer.rb
	app/views/stream_entries/_detailed_status.html.haml
	app/views/stream_entries/_simple_status.html.haml
	config/locales/simple_form.en.yml
2018-02-09 09:25:53 -06:00
Eugen Rochko 219b28e172 Add preference to always display sensitive media (#6448) 2018-02-09 00:26:57 +01:00
David Yip 6d1023b2e9 Merge remote-tracking branch 'tootsuite/master' into merge-upstream
Conflicts:
      app/javascript/styles/mastodon/components.scss
2018-02-02 08:39:52 -06:00
Akihiko Odaki 3cd3cae7f7 Accept ActivityPub announce from the author of the original note (#6236) 2018-02-02 10:22:15 +01:00
codl 2710843f69 remove escapes from server-side γaml handler
this matches a change that was made in the client-side equivalent
2018-01-24 14:23:25 +01:00
David Yip d45ebe26dc Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	app/javascript/styles/mastodon/components.scss
	app/javascript/styles/mastodon/modal.scss
2018-01-19 15:22:10 -06:00
ThibG dc4054abe9 Process mentions and reblogs even from resolved threads (#6299)
This may lead to out-of-order notifications, but this is better than not having
notifications at all.
2018-01-19 19:11:35 +01:00
David Yip 1a7f8eb723 Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	db/schema.rb
2018-01-09 14:16:45 -06:00
Eugen Rochko e19ece354e Skip ActivityPub Announces of non-public objects (#6230)
* Skip ActivityPub Announces of non-public objects

* Skip OStatus reblogs of non-public statuses
2018-01-09 19:35:10 +01:00
Jenkins f9a0b246e5 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-09 05:17:13 +00:00
puckipedia 74e427c24b Add the author of a status to cc if reblogged (#6226)
This makes slightly more sense, and ensures that the author of a post is always referenced in the audience (which some servers might rely on). And the announce is POSTed to the author's inbox anyways.
2018-01-09 00:47:43 +01:00
Jenkins 1f469edac1 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-08 10:17:15 +00:00
Eugen Rochko c8f0be1d6d Revert #5772 (#6221) 2018-01-08 10:57:52 +01:00
Jenkins f10ce3f003 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-08 04:17:11 +00:00
Eugen Rochko d4e0d2563c Fix bad URL schemes being accepted (#6219)
* Fix actors accepting invalid URI schemes or different host between URI and URL

* Fix statuses accepting invalid URI scheme or different host to actor

* Adjust tests to new requirements

* Improve readability of mismatching_origin?/invalid_origin? methods
2018-01-08 05:00:23 +01:00
puckipedia 4ae7b4eb1e Move Article from supported to converted types (#6218) 2018-01-08 00:21:14 +01:00
Jenkins a7b8cb9493 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-03 20:17:13 +00:00
ThibG 5ee5272b69 Don't normalize URLs in toots (#6134)
* Don't normalize URLs in toots

URL normalization is ill-defined and may cause certain links to break.

* Change specs since we are not normalizing user-provided URLs
2018-01-03 20:51:33 +01:00
Akihiko Odaki 7a0479c830 Allow to dereference Follow object for ActivityPub (#5772)
* Allow to dereference Follow object for ActivityPub

* Accept IRI as object representation for Accept activity
2018-01-03 18:08:57 +01:00
Jenkins b42e6973a1 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-03 04:17:11 +00:00
puckipedia 95adc48b92 [!] Sanitize incoming classlist properly (#6162)
* Sanitize classlist properly

* Actually properly sanitize every class after the first

* Improve Formatter spec to check for multiple classes and non-space whitespace
2018-01-03 03:54:08 +01:00
Eugen Rochko 48bc0b8375 Don't leave behind husk of remotely-deleted profile (#6159)
There's no reason for an Account record to persist after Delete->Actor is received. SuspendAccountService is necessary to make sure deleted toots get sent over streaming API properly and home feeds get cleaned up. By removing Account record, we can ensure that if in the future the account is restored remotely (or username reused), it can start with a clean slate.
2018-01-03 00:38:02 +01:00
Eugen Rochko cb45282e47 Fix stats expiring too quickly because of variable mistake (#6155) 2018-01-02 14:02:53 +01:00
David Yip 01f91695f5 Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	app/controllers/auth/confirmations_controller.rb
2017-12-30 17:20:07 -06:00
Eugen Rochko 9427823d5c Add more instance stats APIs (#6125)
* Add GET /api/v1/instance/peers API to reveal known domains

* Add GET /api/v1/instance/activity API

* Make new APIs disableable, exclude private statuses from activity stats

* Fix code style issue

* Fix week timestamps
2017-12-29 19:52:04 +01:00
David Yip f722fcb5a6 Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	app/javascript/images/mastodon-drawer.png
	app/javascript/styles/mastodon/components.scss
2017-12-26 22:24:19 -06:00
Akihiko Odaki d8a0c19166 Fix XML oEmbed support discovery (#6104) 2017-12-27 03:29:49 +01:00
Jenkins 601f0fe4a3 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2017-12-15 01:17:12 +00:00
Eugen Rochko 99a208cd8f Do not hide statuses from silenced accounts from other silenced accounts (#6030) 2017-12-15 01:54:05 +01:00
Jenkins 1366e96a02 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2017-12-13 18:17:16 +00:00
Eugen Rochko 8c04f9417f Fix redundant HTTP request in FetchLinkCardService (#6002) 2017-12-13 12:15:28 +01:00
David Yip a56c9ac5dc Merge remote-tracking branch 'tootsuite/master' into merge-upstream 2017-12-12 02:54:13 -06:00
kibigo! a1c79ef544 Screenshot support for themes 2017-12-10 20:32:28 -08:00
cwm 34630e00cb pulled master, moved locale entry to new location 2017-12-10 15:22:15 -06:00
kibigo! 99e8710095 Javascript intl8n flavour support 2017-12-10 11:08:04 -08:00
abcang a55211ee5b Save media outside transaction (#5959) 2017-12-10 16:33:52 +01:00
cwm 98815c76ce fixed fav setting change 2017-12-09 12:17:20 -06:00
cwm 7ff586054c added a few more things 2017-12-09 11:21:41 -06:00
beatrix f0b37f92a9 Merge pull request #229 from glitch-soc/glitch-theme
Advanced Next-Level Flavours And Skins For Mastodon™
2017-12-06 17:44:07 -05:00
Jenkins a3a7b6a848 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2017-12-06 20:17:13 +00:00