Commit Graph

986 Commits (8a618c8ecac64c0cbd937720bee2d4f13a831477)

Author SHA1 Message Date
Claire 23bce96f0c Change sidekiq-bulk's batch size from 10,000 to 1,000 jobs in one Redis call (#24034) 2023-03-15 03:45:15 +01:00
Jean byroot Boussier 08c2938b4a Upgrade to latest redis-rb 4.x and fix deprecations (#23616)
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2023-03-04 16:38:28 +01:00
Claire fe098d35c2 Fix original account being unfollowed on migration before the follow request could be sent (#21957) 2023-03-03 21:13:55 +01:00
Christian Schmidt cc0c37cd69 Only offer translation for supported languages (#23879) 2023-03-03 21:06:31 +01:00
Claire 7bc43c91a0 Fix “Remove all followers from the selected domains” being more destructive than it claims (#23805) 2023-03-03 20:25:15 +01:00
Nick Schonning 19be243493 Autofix Rubocop Style/HashConversion (#23852) 2023-02-28 22:59:19 +09:00
Nick Schonning 4f6ad6475e Autofix Rubocop Rails/Blank (#23765) 2023-02-22 09:57:56 +09:00
Nick Schonning 247bf6e812 Enable Rubocop Style/StringConcatenation defaults (#23792) 2023-02-22 09:54:36 +09:00
Nick Schonning dd63a919f0 Autofix Rubocop Style/RescueStandardError (#23745) 2023-02-20 11:01:20 +01:00
Nick Schonning e6312104c7 Autofix Rubocop remaining Layout rules (#23679) 2023-02-20 06:58:28 +01:00
Nick Schonning 49f1168050 Autofix Rubocop Style/RedundantBegin (#23703) 2023-02-19 07:09:40 +09:00
Nick Schonning 793f8c7dd5 Autofix Rubocop Style/IfUnlessModifier (#23697) 2023-02-18 12:37:47 +01:00
Nick Schonning a71a0802d6 Autofix Rubocop Lint/SymbolConversion (#23683) 2023-02-18 03:23:49 +01:00
Nick Schonning 31ca76fe2c Run rubocop formatting except line length (#23632) 2023-02-18 06:56:20 +09:00
Claire 0ae0740826 Add API parameter to safeguard unexpect mentions in new posts (#18350) 2023-02-13 16:36:29 +01:00
Claire ea4cffac37 Fix unbounded recursion in post discovery (#23506)
* Add a limit to how many posts can get fetched as a result of a single request

* Add tests

* Always pass `request_id` when processing `Announce` activities

---------

Co-authored-by: nametoolong <nametoolong@users.noreply.github.com>
2023-02-10 22:16:37 +01:00
Nick Schonning e0a02f8696 Apply Rubocop Style/NegatedIfElseCondition (#23451) 2023-02-08 07:07:36 +01:00
Nick Schonning 38ec6d27d8 Autofix Rails/EagerEvaluationLogMessage (#23429)
* Autofix Rails/EagerEvaluationLogMessage

* Update spec for debug block syntax
2023-02-07 03:44:36 +01:00
Claire 2b356a3de6 Fix ActivityPub::ProcessingWorker error on incoming malformed JSON-LD (#23416) 2023-02-06 21:00:58 +01:00
Markus Unterwaditzer 4a902cc7f6 Suppress AddressFamilyError in link verification (#23204)
* Suppress AddressFamilyError

* clarify comment
2023-01-23 13:05:54 +01:00
Claire 8de1412d0f Revert "Remove LDSignature on actor Delete activities (#21466)" (#23185)
This reverts commit 4d9ee0e01d.
2023-01-21 15:33:21 +01:00
Claire b131e01db7 Add more specific error messages to HTTP signature verification (#21617)
* Return specific error on failure to parse Date header

* Add error message when preferredUsername is not set

* Change error report to be JSON and include more details

* Change error report to differentiate unknown account and failed refresh

* Add tests
2023-01-18 16:47:56 +01:00
Claire 42aa864c84 Add support for editing media description and focus point of already-posted statuses (#20878)
* Add backend support for editing media attachments of existing posts

* Allow editing media attachments of already-posted toots

* Add tests
2023-01-18 16:33:55 +01:00
Markus Unterwaditzer b7f37e68ad fix: allow verification when page size exceeds 1MB (using HTML5 parser) (#22879)
* fix: allow verification when page size exceeds 1MB
Truncates the page after 1MB instead

Closes #15316

* switch to HTML5 parser, fix rubocop errors

* undo rubocop fixes

Co-authored-by: Chris Zubak-Skees <chriszs@gmail.com>
2023-01-11 21:59:13 +01:00
Kaspar V 1cd04e5d77 Fix/remove calling private method with send in model (#22951)
* fix(status): remove send usage for private unlink_from_conversations

- make unlink_from_conversations public method
- rename unlink_from_conversations to unlink_from_conversations!
- fix send call on private method in statuses_vacuum and batched_remove_status_service

* fix(feeds_vacuum): replace find_in_batches with in_batches

because active record query results should be a little more efficient than
itterating with map and each. Postgres can grasp such lists of ids much quicker
than ruby can.
Will probably make allmost no difference, but cannot hurt either.
2023-01-11 21:57:24 +01:00
Claire 6ccc1c09b6 Fix possible race conditions when suspending/unsuspending accounts (#22363)
* Fix possible race conditions when suspending/unsuspending accounts

* Fix tests

Tests were assuming SuspensionWorker and UnsuspensionWorker would do the
suspending/unsuspending themselves, but this has changed.
2023-01-05 13:47:21 +01:00
Effy Elden 8b3c8fa218 Be more lenient with OEmbed detection and validation (#22533) 2023-01-05 13:36:24 +01:00
Claire d773490e8a Fix suspension worker crashing on S3-compatible setups without ACL support (#22487) 2022-12-19 17:55:17 +01:00
Claire bd40c1fa5a Ensure exact match is the first result in hashtag searches (#21315)
Fixes #17494
2022-12-15 18:41:40 +01:00
Claire b821678966 Change remote media files to be downloaded outside of transactions (#21796) 2022-12-15 18:09:48 +01:00
Brian Campbell baf43ec2d5 Fix idempotency when database writes are slow (#21840)
There is an idempotency key generated by clients when authoring a post,
and stored in Redis, to ensure that if a user or client retries posting
the same status, we don't get a duplicate.

Hachyderm.io has been experiencing some filesystem and database
performance issues, causing database writes to be slow. This can mean
that there are successful posts, but the reverse proxy returns 504
Gateway Timeout before the idempotency status has been updated; users or
clients who retry (such as Tusky which retries automatically, see
tuskyapp/Tusky#2951) can re-try the same post with the same idempotency
key before it has actually been recorded in Redis, leading to duplicate
posts.

To address this issue, move all of the database updates after the
initial transaction that creates the status into the
`postprocess_status!` method, so we can insert the idempotency key
immediately after the status has been created, significantly reducing
the window in which the status could be created but the idempotency key
not yet stored.

Note: this has not yet been tested; I'm submitting this PR for
discussion and to offer to the Hachyderm.io admins to try out to fix the
multiple posting problem.

Co-authored-by: Brian Campbell <brcampbell@beta.team>
2022-12-15 18:08:40 +01:00
Claire aec7de494f Fix unbounded recursion in account discovery (#22025)
* Fix trying to fetch posts from other users when fetching featured posts

* Rate-limit discovery of new subdomains

* Put a limit on recursively discovering new accounts
2022-12-07 00:15:24 +01:00
Claire 4d9ee0e01d Remove LDSignature on actor Delete activities (#21466)
They are currently not used for anything and represent more than half of the
payload size.
2022-11-27 20:43:24 +01:00
Claire be16e6f26c Fix attachments of edited statuses not being fetched (#21565)
* Fix attachments of edited statuses not being fetched

* Fix tests
2022-11-27 20:39:05 +01:00
Joshua Wood e0940d04f1 Handle links with no href in VerifyLinkService (#20741)
Before this change, the following error would cause VerifyAccountLinksWorker to fail:

NoMethodError: undefined method `downcase' for nil:NilClass
  [PROJECT_ROOT]/app/services/verify_link_service.rb:31 :in `block in link_back_present?`
2022-11-17 10:59:35 +01:00
Hampton Lintorn-Catlin 277e2590e2 Support UTF-8 Characters in Domains During CSV Import (#20592)
* Support UTF-8 Characters in Domains During Import

* Update Changelong
2022-11-14 05:52:13 +01:00
James Tucker c4ff2640c1 Improve performance by avoiding regex construction (#20215)
```ruby
10.times { p /#{FOO}/.object_id }
10.times { p FOO_RE.object_id }
```
2022-11-10 05:49:30 +01:00
Claire 0ac21d1fb8 Fix NameError in Webfinger redirect handling in ActivityPub::FetchRemoteActorService (#20260) 2022-11-09 20:10:38 +01:00
Eugen Rochko 3c80a6e445 Fix being able to spoof link verification (#20217)
- Change verification to happen in `default` queue
- Change verification worker to only be queued if there's something to do
- Add `link` tags from metadata fields to page header of profiles
2022-11-09 08:24:21 +01:00
trwnh 260ea7b222 Change Report category to "violation" if rule IDs are provided (#20137)
* Change Report category to "violation" if rule IDs are provided

* Fix LiteralAsCondition

* Add parentheses to conditional statement
2022-11-08 17:28:02 +01:00
Claire 5778ba1028 Fix validation error in SynchronizeFeaturedTagsCollectionWorker (#20018)
* Fix followers count not being updated when migrating follows

Fixes #19900

* Fix validation error in SynchronizeFeaturedTagsCollectionWorker

Also saves remote user's chosen case for hashtags

* Limit remote featured tags before validation
2022-11-07 22:35:53 +01:00
Eugen Rochko 2e68cbe212 Fix n+1 query during status removal (#19753) 2022-11-05 00:09:52 +01:00
Claire 4b0e72b2b7 Fix reblogs being discarded after the reblogged status (#19731) 2022-11-04 16:31:44 +01:00
Eugen Rochko 5b639f9e15 Add caching for payload serialization during fan-out (#19642) 2022-11-04 13:21:06 +01:00
Claire ae89367588 Change mentions of blocked users to not be processed (#19725)
Fixes #19698
2022-11-04 13:19:12 +01:00
Claire 208ae8a4e0 Fix bookmark import stopping at the first failure (#19669)
Fixes #19389
2022-11-02 16:38:23 +01:00
Eugen Rochko 6099eecf57 Fix account migration form ever using outdated account data (#18429) 2022-10-29 01:31:45 +02:00
Eugen Rochko b121befc33 Fix wrong math function used in search query (#19481) 2022-10-27 02:10:38 +02:00
Eugen Rochko 821c050f08 Fix negatives values in search index causing queries to fail (#19464) 2022-10-26 13:00:43 +02:00
Eugen Rochko 6c1c7d9d3b Change unauthenticated search to not support pagination in REST API (#19326)
- Only exact search matches for queries with < 5 characters
- Do not support queries with `offset` (pagination)
- Return HTTP 401 on truthy `resolve` instead of overriding to false
2022-10-26 12:10:02 +02:00