Commit Graph

1090 Commits (2fe1b8d1695d8faa452a69872fde94ccc4611576)

Author SHA1 Message Date
Eugen Rochko 67166de865
Add `from:me` syntax to search () 2023-08-28 12:43:00 +02:00
jsgoldstein 8e8747c564
Update ordering to use `id` from body of document instead of deprecated `_id` () 2023-08-25 18:16:08 +02:00
Claire 82ec6f162b
Fix statuses search Elasticsearch query () 2023-08-25 17:01:56 +02:00
jsgoldstein 30c191aaa0
Add new public status index ()
Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-08-24 16:40:04 +02:00
jsgoldstein 85057865b4
Update Account Search to prioritize username over display name () 2023-08-23 15:40:09 +02:00
Robert R George cf6f70799b
Add support for federating `memorial` attribute () 2023-08-23 08:27:24 +02:00
Claire 90ec88d58b
Add support for `indexable` attribute on remote actors ()
Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2023-08-14 18:54:51 +02:00
Eugen Rochko 72423bc8f6
Change account search tokenizer and queries () 2023-08-08 09:09:14 +02:00
Claire 30c64bf616
Fix list import concurrently creating lists of the same name () 2023-08-07 17:59:20 +02:00
Claire 6b896b20cc
Add primary key to preview_cards_statuses join table (includes deduplication migration) () 2023-08-03 11:12:52 +02:00
Emelia Smith e258b4cb64
Refactor: replace whitelist_mode mentions with limited_federation_mode () 2023-08-02 19:32:48 +02:00
Christian Schmidt f96e4b3047
Use original URL in preview if it redirects to 4xx page () 2023-07-28 23:02:08 +02:00
Misty De Méo 12a6cf569e
Storage: add :azure to remaining callers () 2023-07-27 16:13:45 +02:00
Claire 1e3b19230a
Add stricter protocol fields validation for accounts () 2023-07-20 18:23:48 +02:00
Jeong Arm 664b0ca8cb
Check if json body is null on Activitipub::ProcessingWorker () 2023-07-17 15:51:30 +02:00
Matt Jankowski ce43ed144c
Rails 7.0 update () 2023-07-13 09:36:07 +02:00
Matt Jankowski 2e1391fdd2
Fix `Naming/MemoizedInstanceVariableName` cop () 2023-07-12 10:08:51 +02:00
jsgoldstein 99be47f8b9
Change searching with # to include account index () 2023-07-10 20:58:13 +02:00
Claire 999c343946
Fix remote accounts being possibly persisted to database with incomplete protocol values () 2023-07-10 18:42:19 +02:00
Claire c27b82a437
Add `forward_to_domains` parameter to `POST /api/v1/reports` () 2023-07-10 18:26:56 +02:00
Matt Jankowski f3fca78756
Refactor `NotificationMailer` to use parameterization () 2023-07-10 03:06:22 +02:00
Matt Jankowski cf33028f35
Admin mailer parameterization () 2023-07-08 20:03:38 +02:00
Eugen Rochko 93e8a15415
Add forwarding of reported replies to servers being replied to () 2023-07-08 20:00:02 +02:00
Daniel M Brasil 383c00819c
Fix `/api/v2/search` not working with following query param () 2023-07-03 18:06:57 +02:00
Eugen Rochko ba06a2f104
Revert "Rails 7 update" () 2023-07-02 11:14:22 +02:00
Matt Jankowski 50c2a03695
Rails 7 update () 2023-07-02 10:38:53 +02:00
Claire a209d1e683
Fix ResolveURLService not resolving local URLs for remote content () 2023-06-29 14:48:54 +02:00
jsgoldstein 4581a528f7
Change account search to match by text when opted-in ()
Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2023-06-29 13:05:21 +02:00
Eugen Rochko bb4756c823
Change files to be deleted in batches instead of one-by-one () 2023-06-26 14:17:41 +02:00
Emelia Smith 3a91603b15
Prevent UserCleanupScheduler from overwhelming streaming () 2023-06-20 18:04:35 +02:00
Eugen Rochko 4c9406bdb0
Add time zone preference () 2023-06-10 03:29:37 +02:00
Nick Schonning c66250abf1
Autofix Rubocop Regex Style rules ()
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-06-06 14:50:51 +02:00
Claire 8884d1ece0
Add support for importing lists () 2023-06-01 14:47:31 +02:00
Christian Schmidt 69057467cb
Translate CW, poll options and media descriptions ()
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-06-01 00:10:21 +02:00
S.H 8c183a9831
Align argument type to keyword argument for private process method in FetchResourceService () 2023-05-30 09:28:01 +02:00
Matt Jankowski 9f5deb310b
Fix Performance/MapCompact cop ()
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-05-23 10:49:12 +02:00
Essem 5fd8d1e417
Fix oversight in backup service () 2023-05-19 11:27:10 +02:00
Matt Jankowski 2c6c398c60
Fix Performance/CollectionLiteralInLoop cop () 2023-05-04 05:33:55 +02:00
Claire a2a22bad23
Fix various edge cases with local moves () 2023-05-03 19:19:25 +02:00
Claire c98b012583
Change Move handler to also move list memberships () 2023-05-03 14:03:38 +02:00
Matt Jankowski 668a19a2f3
Fix Performance/DeletePrefix cop () 2023-05-02 21:07:45 +02:00
Matt Jankowski f1c1dd0118
Rename `with_lock` to `with_redis_lock` to avoid confusion with ActiveRecord's method () 2023-05-02 18:16:07 +02:00
Claire a89b02af92
Check domain suspensions before trying to fetch resources () 2023-05-02 15:22:19 +02:00
Claire b233da5996
Optimize archive export service and export zip files instead of gzipped tar files () 2023-05-02 15:09:43 +02:00
Claire 32a030dd74
Rewrite import feature () 2023-05-02 12:08:48 +02:00
Matt Jankowski d902a707a3
Fix Rails/CompactBlank cop () 2023-04-30 14:07:21 +02:00
Nick Schonning 49fad26eca
Drop EOL Ruby 2.7 () 2023-04-27 01:46:18 +02:00
Matt Jankowski 0a5f0a8b20
Remove instance variables from helper usage () 2023-04-23 22:35:54 +02:00
Claire 85b1b45820
Fix crash in NotifyService when trying to send an email notification for polls () 2023-04-17 13:13:36 +02:00
Claire 99e3e152cd
Fix crash in NotifyService when trying to send an email notification for post edits () 2023-04-08 12:51:14 +02:00
Eugen Rochko a9b5598c97
Change user settings to be stored in a more optimal way ()
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-03-30 14:44:00 +02:00
Christian Schmidt eb38e9df31
Requeue expiration notification () 2023-03-29 10:52:40 +02:00
Takeshi Umeda 38c84f57b6
Refactoring relations_map () 2023-03-21 10:32:58 +01:00
Christian Schmidt bd047acc35
Replace `Status#translatable?` with language matrix in separate endpoint () 2023-03-16 11:07:24 +01:00
Claire 1d0ad558ff
Change sidekiq-bulk's batch size from 10,000 to 1,000 jobs in one Redis call () 2023-03-15 03:45:15 +01:00
Jean byroot Boussier 922837dc96
Upgrade to latest redis-rb 4.x and fix deprecations ()
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2023-03-04 16:38:28 +01:00
Claire 050f1669c6
Fix original account being unfollowed on migration before the follow request could be sent () 2023-03-03 21:13:55 +01:00
Christian Schmidt 5a8c651e8f
Only offer translation for supported languages () 2023-03-03 21:06:31 +01:00
Claire c2a046ded1
Fix “Remove all followers from the selected domains” being more destructive than it claims () 2023-03-03 20:25:15 +01:00
Nick Schonning 56489cdc4f
Autofix Rubocop Style/HashConversion () 2023-02-28 22:59:19 +09:00
Nick Schonning 4ff44be134
Autofix Rubocop Rails/Blank () 2023-02-22 09:57:56 +09:00
Nick Schonning 0cfdd1a401
Enable Rubocop Style/StringConcatenation defaults () 2023-02-22 09:54:36 +09:00
Nick Schonning 59c8d43d94
Autofix Rubocop Style/RescueStandardError () 2023-02-20 11:01:20 +01:00
Nick Schonning 717683d1c3
Autofix Rubocop remaining Layout rules () 2023-02-20 06:58:28 +01:00
Nick Schonning 2177daeae9
Autofix Rubocop Style/RedundantBegin () 2023-02-19 07:09:40 +09:00
Nick Schonning e2a3ebb271
Autofix Rubocop Style/IfUnlessModifier () 2023-02-18 12:37:47 +01:00
Nick Schonning 634368c491
Autofix Rubocop Lint/SymbolConversion () 2023-02-18 03:23:49 +01:00
Nick Schonning 669f6d2c0a
Run rubocop formatting except line length () 2023-02-18 06:56:20 +09:00
Claire d6930b3847
Add API parameter to safeguard unexpect mentions in new posts () 2023-02-13 16:36:29 +01:00
Claire 0c9eac80d8
Fix unbounded recursion in post discovery ()
* Add a limit to how many posts can get fetched as a result of a single request

* Add tests

* Always pass `request_id` when processing `Announce` activities

---------

Co-authored-by: nametoolong <nametoolong@users.noreply.github.com>
2023-02-10 22:16:37 +01:00
Nick Schonning f68bb52556
Apply Rubocop Style/NegatedIfElseCondition () 2023-02-08 07:07:36 +01:00
Nick Schonning ed570050c6
Autofix Rails/EagerEvaluationLogMessage ()
* Autofix Rails/EagerEvaluationLogMessage

* Update spec for debug block syntax
2023-02-07 03:44:36 +01:00
Claire bb89f0af8a
Fix ActivityPub::ProcessingWorker error on incoming malformed JSON-LD () 2023-02-06 21:00:58 +01:00
Markus Unterwaditzer f2a6e71bb6
Suppress AddressFamilyError in link verification ()
* Suppress AddressFamilyError

* clarify comment
2023-01-23 13:05:54 +01:00
Claire 628dcbb732
Revert "Remove LDSignature on actor Delete activities ()" ()
This reverts commit f4f2b062ec.
2023-01-21 15:33:21 +01:00
Claire 68dcbcb7bf
Add more specific error messages to HTTP signature verification ()
* Return specific error on failure to parse Date header

* Add error message when preferredUsername is not set

* Change error report to be JSON and include more details

* Change error report to differentiate unknown account and failed refresh

* Add tests
2023-01-18 16:47:56 +01:00
Claire 4b92e59f4f
Add support for editing media description and focus point of already-posted statuses ()
* Add backend support for editing media attachments of existing posts

* Allow editing media attachments of already-posted toots

* Add tests
2023-01-18 16:33:55 +01:00
Markus Unterwaditzer 0c689b9d01
fix: allow verification when page size exceeds 1MB (using HTML5 parser) ()
* fix: allow verification when page size exceeds 1MB
Truncates the page after 1MB instead

Closes 

* switch to HTML5 parser, fix rubocop errors

* undo rubocop fixes

Co-authored-by: Chris Zubak-Skees <chriszs@gmail.com>
2023-01-11 21:59:13 +01:00
Kaspar V ae62e5fa53
Fix/remove calling private method with send in model ()
* fix(status): remove send usage for private unlink_from_conversations

- make unlink_from_conversations public method
- rename unlink_from_conversations to unlink_from_conversations!
- fix send call on private method in statuses_vacuum and batched_remove_status_service

* fix(feeds_vacuum): replace find_in_batches with in_batches

because active record query results should be a little more efficient than
itterating with map and each. Postgres can grasp such lists of ids much quicker
than ruby can.
Will probably make allmost no difference, but cannot hurt either.
2023-01-11 21:57:24 +01:00
Claire 18fb01ef7c
Fix possible race conditions when suspending/unsuspending accounts ()
* Fix possible race conditions when suspending/unsuspending accounts

* Fix tests

Tests were assuming SuspensionWorker and UnsuspensionWorker would do the
suspending/unsuspending themselves, but this has changed.
2023-01-05 13:47:21 +01:00
Effy Elden 4dc9152b3e
Be more lenient with OEmbed detection and validation () 2023-01-05 13:36:24 +01:00
Claire c1de673060
Fix suspension worker crashing on S3-compatible setups without ACL support () 2022-12-19 17:55:17 +01:00
Claire bf1c7e2122
Ensure exact match is the first result in hashtag searches ()
Fixes 
2022-12-15 18:41:40 +01:00
Claire 2644a28cb3
Change remote media files to be downloaded outside of transactions () 2022-12-15 18:09:48 +01:00
Brian Campbell 2d12948220
Fix idempotency when database writes are slow ()
There is an idempotency key generated by clients when authoring a post,
and stored in Redis, to ensure that if a user or client retries posting
the same status, we don't get a duplicate.

Hachyderm.io has been experiencing some filesystem and database
performance issues, causing database writes to be slow. This can mean
that there are successful posts, but the reverse proxy returns 504
Gateway Timeout before the idempotency status has been updated; users or
clients who retry (such as Tusky which retries automatically, see
) can re-try the same post with the same idempotency
key before it has actually been recorded in Redis, leading to duplicate
posts.

To address this issue, move all of the database updates after the
initial transaction that creates the status into the
`postprocess_status!` method, so we can insert the idempotency key
immediately after the status has been created, significantly reducing
the window in which the status could be created but the idempotency key
not yet stored.

Note: this has not yet been tested; I'm submitting this PR for
discussion and to offer to the Hachyderm.io admins to try out to fix the
multiple posting problem.

Co-authored-by: Brian Campbell <brcampbell@beta.team>
2022-12-15 18:08:40 +01:00
Claire c8849d6cee
Fix unbounded recursion in account discovery ()
* Fix trying to fetch posts from other users when fetching featured posts

* Rate-limit discovery of new subdomains

* Put a limit on recursively discovering new accounts
2022-12-07 00:15:24 +01:00
Claire f4f2b062ec
Remove LDSignature on actor Delete activities ()
They are currently not used for anything and represent more than half of the
payload size.
2022-11-27 20:43:24 +01:00
Claire 625216d8e1
Fix attachments of edited statuses not being fetched ()
* Fix attachments of edited statuses not being fetched

* Fix tests
2022-11-27 20:39:05 +01:00
Joshua Wood daf6f3453e
Handle links with no href in VerifyLinkService ()
Before this change, the following error would cause VerifyAccountLinksWorker to fail:

NoMethodError: undefined method `downcase' for nil:NilClass
  [PROJECT_ROOT]/app/services/verify_link_service.rb:31 :in `block in link_back_present?`
2022-11-17 10:59:35 +01:00
Hampton Lintorn-Catlin 147d8bd8fc
Support UTF-8 Characters in Domains During CSV Import ()
* Support UTF-8 Characters in Domains During Import

* Update Changelong
2022-11-14 05:52:13 +01:00
James Tucker 78a6b871fe
Improve performance by avoiding regex construction ()
```ruby
10.times { p /#{FOO}/.object_id }
10.times { p FOO_RE.object_id }
```
2022-11-10 05:49:30 +01:00
Claire a5394980f2
Fix NameError in Webfinger redirect handling in ActivityPub::FetchRemoteActorService () 2022-11-09 20:10:38 +01:00
Eugen Rochko e98833748e
Fix being able to spoof link verification ()
- Change verification to happen in `default` queue
- Change verification worker to only be queued if there's something to do
- Add `link` tags from metadata fields to page header of profiles
2022-11-09 08:24:21 +01:00
trwnh b1a48e05b6
Change Report category to "violation" if rule IDs are provided ()
* Change Report category to "violation" if rule IDs are provided

* Fix LiteralAsCondition

* Add parentheses to conditional statement
2022-11-08 17:28:02 +01:00
Claire bbf74498f5
Fix validation error in SynchronizeFeaturedTagsCollectionWorker ()
* Fix followers count not being updated when migrating follows

Fixes 

* Fix validation error in SynchronizeFeaturedTagsCollectionWorker

Also saves remote user's chosen case for hashtags

* Limit remote featured tags before validation
2022-11-07 22:35:53 +01:00
Eugen Rochko c4b92b1aee
Fix n+1 query during status removal () 2022-11-05 00:09:52 +01:00
Claire c2170991c7
Fix reblogs being discarded after the reblogged status () 2022-11-04 16:31:44 +01:00
Eugen Rochko 5f9e47be34
Add caching for payload serialization during fan-out () 2022-11-04 13:21:06 +01:00
Claire 4fb0aae636
Change mentions of blocked users to not be processed ()
Fixes 
2022-11-04 13:19:12 +01:00
Claire e0eb39d41b
Fix bookmark import stopping at the first failure ()
Fixes 
2022-11-02 16:38:23 +01:00