Also:
- Fix locks not being removed when jobs go to the dead job queue
- Add UI for managing locks to the Sidekiq dashboard
- Remove unused Sidekiq workers
Fix#13349
* Add announcements
Fix#11006
* Add reactions to announcements
* Add admin UI for announcements
* Add unit tests
* Fix issues
- Add `with_dismissed` param to announcements API
- Fix end date not being formatted when time range is given
- Fix announcement delete causing reactions to send streaming updates
- Fix announcements container growing too wide and mascot too small
- Fix `all_day` being settable when no time range is given
- Change text "Update" to "Announcement"
* Fix scheduler unpublishing announcements before they are due
* Fix filter params not being passed to announcements filter
* Fix wrong grouping in Twitter valid_url regex
* Add support for xmpp URIs
Fixes#9776
The difficult part is autolinking, because Twitter-text's extractor does
some pretty ad-hoc stuff to find things that “look like” URLs, and XMPP
URIs do not really match the assumptions of that lib, so it doesn't sound
wise to try to shoehorn it into the existing regex.
This is why I used a specific regex (very close, although slightly more
permissive than the RFC), and a specific scan function (a simplified version
of the generalized one from Twitter).
* Remove leading “xmpp:” from auto-linked text
When authenticating via OAuth, the resource owner password grant
strategy is allowed by Mastodon, but (without this PR), it does not
attempt to authenticate against LDAP or PAM. As a result, LDAP or PAM
authenticated users cannot sign in to Mastodon with their
email/password credentials via OAuth (for instance, for native/mobile
app users).
This PR fleshes out the authentication strategy supplied to doorkeeper
in its initializer by looking up the user with LDAP and/or PAM when
devise is configured to use LDAP/PAM backends. It attempts to follow the
same logic as the Auth::SessionsController for handling email/password
credentials.
Note #1: Since this pull request affects an initializer, it's unclear
how to add test automation.
Note #2: The PAM authentication path has not been manually tested. It
was added for completeness sake, and it is hoped that it can be manually
tested before merging.
* Add backend support for bookmarks
Bookmarks behave like favourites, except they aren't shared with other
users and do not have an associated counter.
* Add spec for bookmark endpoints
* Add front-end support for bookmarks
* Introduce OAuth scopes for bookmarks
* Add bookmarks to archive takeout
* Fix migration
* Coding style fixes
* Fix rebase issue
* Update bookmarked_statuses to latest UI changes
* Update bookmark actions to properly reflect status changes in state
* Add bookmarks item to single-column layout
* Make active bookmarks red
Change the behaviour of remotable concern. Previously, it would skip
downloading an attachment if the stored remote URL is identical to
the new one. Now it would not be skipped if the attachment is not
actually currently stored by Paperclip.
The default limit of 10 retries with exponential backoff meant
that if the S3 server was timing out, you would be stuck with it
for much, much longer than the 5 second read timeout we expect.
The uploading happens within a database transaction, which means
a failing S3 server could negatively affect database performance
It's possible that after commit callbacks were not firing when
exceptions occurred in the process. Also, the default Sidekiq
strategy does not push indexing jobs immediately, which is not
necessary and could be part of the issue too.
* Add nodeinfo endpoint
* dont commit stuff from my local dev
* consistant naming since we implimented 2.1 schema
* Add some additional node info stuff
* Add nodeinfo endpoint
* dont commit stuff from my local dev
* consistant naming since we implimented 2.1 schema
* expanding this to include federation info
* codeclimate feedback
* CC feedback
* using activeserializers seems like a good idea...
* get rid of draft 2.1 version
* Reimplement 2.1, also fix metaData -> metadata
* Fix metaData -> metadata here too
* Fix nodeinfo 2.1 tests
* Implement cache for monthly user aggregate
* Useless
* Remove ostatus from the list of supported protocols
* Fix nodeinfo's open_registration reading obsolete setting variable
* Only serialize domain blocks with user-facing limitations
* Do not needlessly list noop severity in nodeinfo
* Only serialize domain blocks info in nodeinfo when they are set to be displayed to everyone
* Enable caching for nodeinfo endpoints
* Fix rendering nodeinfo
* CodeClimate fixes
* Please CodeClimate
* Change InstancePresenter#active_user_count_months for clarity
* Refactor NodeInfoSerializer#metadata
* Remove nodeinfo 2.1 support as the schema doesn't exist
* Clean-up
The instrumentation code was used for StatsD metrics collection
prior to the switch to the nsa gem and should have been removed
at that point as it no longer does anything at all
* Rate limit based on remote address IP, not on potential reverse proxy
* Limit rate of unauthenticated API requests further
* Rate-limit paging requests to one every 3 seconds
Deletions take a lot of resources to execute and cause a lot of
federation traffic, so it makes sense to decrease the number
someone can queue up through the API.
30 per 30 minutes
I also added "public" here, as I can't think of a good reason not to add it. Perhaps it has some marginal benefit in that ISPs (or other proxies) can cache it for all users. The assets are certainly publicly available and the same for all users.
* Add REST API for creating an account
The method is available to apps with a token obtained via the client
credentials grant. It creates a user and account records, as well as
an access token for the app that initiated the request. The user is
unconfirmed, and an e-mail is sent as usual.
The method returns the access token, which the app should save for
later. The REST API is not available to users with unconfirmed
accounts, so the app must be smart to wait for the user to click a
link in their e-mail inbox.
The method is rate-limited by IP to 5 requests per 30 minutes.
* Redirect users back to app from confirmation if they were created with an app
* Add tests
* Return 403 on the method if registrations are not open
* Require agreement param to be true in the API when creating an account
Right now, this includes three endpoints: host-meta, webfinger, and change-password.
host-meta and webfinger are publicly available and do not use any authentication. Nothing bad can be done by accessing them in a user's browser.
change-password being CORS-enabled will only reveal the URL it redirects to (which is /auth/edit) but not anything about the actual /auth/edit page, because it does not have CORS enabled.
The documentation for hosting an instance on a different domain should also be updated to point out that Access-Control-Allow-Origin: * should be set at a minimum for the /.well-known/host-meta redirect to allow browser-based non-proxied instance discovery.
* Verify link ownership with rel="me"
* Add explanation about verification to UI
* Perform link verifications
* Add click-to-copy widget for verification HTML
* Redesign edit profile page
* Redesign forms
* Improve responsive design of settings pages
* Restore landing page sign-up form
* Fix typo
* Support <link> tags, add spec
* Fix links not being verified on first discovery and passive updates
CSFR-prevention is already implemented but adding this doesn't hurt.
A brief introduction to Same-Site cookies (and the difference between strict and
lax) can be found at
https://blog.mozilla.org/security/2018/04/24/same-site-cookies-in-firefox-60/
TLDR: We use lax since we want the cookies to be sent when the user navigates
safely from an external site.
* Fix uncaching worker
* Revert to using Paperclip's filesystem backend instead of fog-local
fog-local has lots of concurrency issues, causing failure to delete files,
dangling file records, and spurious errors UncacheMediaWorker
Adopted from GitLab CE. Generate new migration with:
rails g post_deployment_migration name_of_migration_here
By default they are run together with db:migrate. To not run them,
the env variable SKIP_POST_DEPLOYMENT_MIGRATIONS must be set
Code by Yorick Peterse <yorickpeterse@gmail.com>, see also:
83c8241160
* Add more granular OAuth scopes
* Add human-readable descriptions of the new scopes
* Ensure new scopes look good on the app UI
* Add tests
* Group scopes in screen and color-code dangerous ones
* Fix wrong extra scope
If Mastodon accesses to the hidden service via transparent proxy, it's needed to avoid checking whether it's a private address, since `.onion` is resolved to a private address.
I was previously using the `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` to provide that function. However, I realized that using `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` is redundant, since this specification is always used with `ALLOW_ACCESS_TO_HIDDEN_SERVICE`. Therefore, I decided to integrate the setting of `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` into` ALLOW_ACCESS_TO_HIDDEN_SERVICE`.
- POST /api/v1/push/subscription
- PUT /api/v1/push/subscription
- DELETE /api/v1/push/subscription
- New OAuth scope: "push" (required for the above methods)
The previous rate limit allowed to post media so fast that it is possible
to fill up the disk space even before an administrator notices. The new
rate limit is configured so that it takes 24 hours to eat 10 gigabytes:
10 * 1024 / 8 / (24 * 60 / 30) = 27 (which rounded to 30)
The period is set long so that it does not prevent from attaching several
media to one post, which would happen in a short period. For example,
if the period is 5 minutes, the rate limit would be:
10 * 1024 / 8 / (24 * 60 / 5) = 4
This long period allows to lift the limit up.
* No need to re-require sidekiq plugins, they are required via Gemfile
* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile
* Replace ruby-oembed with FetchOEmbedService
Reduce startup by 45382 allocated objects
* Remove preloaded JSON-LD in favour of caching HTTP responses
Reduce boot RAM by about 6 MiB
* Fix tests
* Fix test suite by stubbing out JSON-LD contexts
* Add support for HTTP client proxy
* Add access control for darknet
Supress error when access to darknet via transparent proxy
* Fix the codes pointed out
* Lint
* Fix an omission + lint
* any? -> include?
* Change detection method to regexp to avoid test fail
Eugen Rochko