Commit Graph

246 Commits (db21a6c5ee2586cf483193e0e6d606629140546a)

Author SHA1 Message Date
Eugen Rochko 9bf5658885 Add tight rate-limit for API deletions (#10042)
Deletions take a lot of resources to execute and cause a lot of
federation traffic, so it makes sense to decrease the number
someone can queue up through the API.

30 per 30 minutes
2019-02-14 06:27:54 +01:00
Eugen Rochko 5adc3f5676 Fix URL linkifier grabbing full-width spaces and quotations (#9997)
Fix #9993
Fix #5654
2019-02-09 20:13:11 +01:00
Eugen Rochko 14d49e814a Add timeouts for S3 (#9842) 2019-01-18 01:36:59 +01:00
Moritz Heiber f2cefd9d73 Disable Same-Site cookie implementation to fix SSO issues on WebKit browsers (#9819) 2019-01-15 23:11:46 +01:00
Nolan Lawson 4fa76b21b6 Enable immutable caching for S3 objects (#9722)
I also added "public" here, as I can't think of a good reason not to add it. Perhaps it has some marginal benefit in that ISPs (or other proxies) can cache it for all users. The assets are certainly publicly available and the same for all users.
2019-01-05 12:29:53 -05:00
Eugen Rochko bbf9f4f93b Add REST API for creating an account (#9572)
* Add REST API for creating an account

The method is available to apps with a token obtained via the client
credentials grant. It creates a user and account records, as well as
an access token for the app that initiated the request. The user is
unconfirmed, and an e-mail is sent as usual.

The method returns the access token, which the app should save for
later. The REST API is not available to users with unconfirmed
accounts, so the app must be smart to wait for the user to click a
link in their e-mail inbox.

The method is rate-limited by IP to 5 requests per 30 minutes.

* Redirect users back to app from confirmation if they were created with an app

* Add tests

* Return 403 on the method if registrations are not open

* Require agreement param to be true in the API when creating an account
2018-12-24 19:12:38 +01:00
Eugen Rochko ab443aaba8 Skip mailer job retries when a record no longer exists (#9590)
Fix #8666
2018-12-21 06:16:17 +01:00
ThibG fe40452a36 Use same CORS policy for /@:username and /users/:username (#9485)
Fixes #8189

rack-cors being called before the application router, it does not follow
the redirection, and we need a separate rule for /users/:username.
2018-12-10 21:39:47 +01:00
ThibG 4813053e97 Preload common JSON-LD contexts (#9412)
Fixes #9411
2018-12-02 16:46:13 +01:00
Ben Lubar e2b221d9f1 Allow cross-origin requests to /.well-known/* URLs. (#9083)
Right now, this includes three endpoints: host-meta, webfinger, and change-password.

host-meta and webfinger are publicly available and do not use any authentication. Nothing bad can be done by accessing them in a user's browser.

change-password being CORS-enabled will only reveal the URL it redirects to (which is /auth/edit) but not anything about the actual /auth/edit page, because it does not have CORS enabled.

The documentation for hosting an instance on a different domain should also be updated to point out that Access-Control-Allow-Origin: * should be set at a minimum for the /.well-known/host-meta redirect to allow browser-based non-proxied instance discovery.
2018-10-25 03:13:35 +02:00
Eugen Rochko e3b2234382 Add unread indicator to conversations (#9009) 2018-10-19 01:47:29 +02:00
ThibG f8e9555e73 Add manifest_src to CSP, add blob to connect_src (#8967) 2018-10-12 19:07:30 +02:00
Eugen Rochko 0dbb3a8786 Fix CSP headers blocking media and development environment (#8962)
Regression from #8957
2018-10-12 01:43:09 +02:00
ThibG 51c53e709f Set Content-Security-Policy rules through RoR's config (#8957)
* Set CSP rules in RoR's configuration

* Override CSP setting in the embed controller to allow frames
2018-10-11 20:35:46 +02:00
Sascha b7a337f63c add ffmpeg initializer (#8855)
* add ffmpeg initializer

* use different expression to check for environment var
2018-10-09 03:02:52 +02:00
ashleyhull-versent 00d31a292c rubocop issues - Cleaning up (#8912)
* cleanup pass

* undo mistakes

* fixed.

* revert
2018-10-08 04:50:11 +02:00
aus-social c883b1ffc9 lint pass 2 (#8878)
* Code quality pass

* Typofix

* Update applications_controller_spec.rb

* Update applications_controller_spec.rb
2018-10-04 17:38:04 +02:00
aus-social a53bcb6213 Lint pass (#8876) 2018-10-04 12:36:53 +02:00
Yamagishi Kazutoshi 78558e5e1c Fix that Rails.cache information could not be sent via StatsD (#8831) 2018-09-30 00:05:59 +02:00
Eugen Rochko d3105031f8 Redesign forms, verify link ownership with rel="me" (#8703)
* Verify link ownership with rel="me"

* Add explanation about verification to UI

* Perform link verifications

* Add click-to-copy widget for verification HTML

* Redesign edit profile page

* Redesign forms

* Improve responsive design of settings pages

* Restore landing page sign-up form

* Fix typo

* Support <link> tags, add spec

* Fix links not being verified on first discovery and passive updates
2018-09-18 16:45:58 +02:00
luzpaz 1bce70d3c7 Misc. typos (#8694)
Found via `codespell -q 3 --skip="./app/javascript/mastodon/locales,./config/locales"`
2018-09-14 00:53:09 +02:00
Sorin Davidoi 0b1c90bf88 feat(cookies): Use the same-site attribute to lax (#8626)
CSFR-prevention is already implemented but adding this doesn't hurt.

A brief introduction to Same-Site cookies (and the difference between strict and
lax) can be found at
https://blog.mozilla.org/security/2018/04/24/same-site-cookies-in-firefox-60/

TLDR: We use lax since we want the cookies to be sent when the user navigates
safely from an external site.
2018-09-08 23:54:28 +02:00
M Somerville 4b27569841 Rename S3_CLOUDFRONT_HOST to S3_ALIAS_HOST. (#8423)
Still check for S3_CLOUDFRONT_HOST for existing installs.
2018-08-25 13:27:08 +02:00
ThibG 97f2dc6761 Revert to using Paperclip's filesystem storage, and fix dangling records in remove_remote (#8339)
* Fix uncaching worker

* Revert to using Paperclip's filesystem backend instead of fog-local

fog-local has lots of concurrency issues, causing failure to delete files,
dangling file records, and spurious errors UncacheMediaWorker
2018-08-21 17:53:01 +02:00
Immae cbaabe0215 Add ldap search filter (#8151) 2018-08-15 18:12:44 +02:00
Eugen Rochko 98184a889e Add post-deployment migration system (#8182)
Adopted from GitLab CE. Generate new migration with:

    rails g post_deployment_migration name_of_migration_here

By default they are run together with db:migrate. To not run them,
the env variable SKIP_POST_DEPLOYMENT_MIGRATIONS must be set

Code by Yorick Peterse <yorickpeterse@gmail.com>, see also:

83c8241160
2018-08-13 13:40:01 +02:00
abcang b46416fe47 Add secure option to additional cookie (#8069) 2018-07-25 18:49:47 +02:00
Eugen Rochko 34fdf77f48 Add more granular OAuth scopes (#7929)
* Add more granular OAuth scopes

* Add human-readable descriptions of the new scopes

* Ensure new scopes look good on the app UI

* Add tests

* Group scopes in screen and color-code dangerous ones

* Fix wrong extra scope
2018-07-05 18:31:35 +02:00
MIYAGI Hikaru ac56fa3c22 Merge `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` into `ALLOW_ACCESS_TO_HIDDEN_SERVICE` (#7901)
If Mastodon accesses to the hidden service via transparent proxy, it's needed to avoid checking whether it's a private address, since `.onion` is resolved to a private address.
I was previously using the `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` to provide that function. However, I realized that using `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` is redundant, since this specification is always used with `ALLOW_ACCESS_TO_HIDDEN_SERVICE`. Therefore, I decided to integrate the setting of `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` into` ALLOW_ACCESS_TO_HIDDEN_SERVICE`.
2018-06-29 15:36:02 +02:00
Eugen Rochko f8bc071e03 Add dat, dweb, ipfs, ipns, ssb, gopher protocols to URL extractor (#7810)
* Add dat:// and gopher:// to URL extractor

Fix #6072

* Fix comment indent

* Add dweb, ipfs, ipns, ssb
2018-06-15 20:21:47 +02:00
Eugen Rochko 9ed3212e35 Remove rack-timeout (#7809)
Timeout considered harmful due to leaving the app in a broken
state, including unreaped database connections
2018-06-15 19:46:25 +02:00
Eugen Rochko b416ba6692 Disable AMS logging (#7623)
Especially in production it's just noise and doesn't mix well with the log format
2018-05-26 01:08:31 +02:00
MIYAGI Hikaru 7dbf83d6b8 User agent for WebFinger (#7531)
* User agent for WebFinger

* local_domain → web_domain

* 'http' is away accidentally...
2018-05-18 01:47:22 +02:00
Eugen Rochko e86a4fe36b Add REST API for Web Push Notifications subscriptions (#7445)
- POST /api/v1/push/subscription
- PUT /api/v1/push/subscription
- DELETE /api/v1/push/subscription
- New OAuth scope: "push" (required for the above methods)
2018-05-11 11:49:12 +02:00
Hugo Gameiro 27cfb13b83 Improve OpenStack v3 compatibility (#7392)
* Update paperclip.rb

* Update .env.production.sample

* Update paperclip.rb
2018-05-07 02:28:28 +02:00
Akihiko Odaki 854d974499 Add a missing question mark in rack_attack.rb (#7338) 2018-05-03 18:51:00 +02:00
Akihiko Odaki 5cddff0795 Throttle media post (#7337)
The previous rate limit allowed to post media so fast that it is possible
to fill up the disk space even before an administrator notices. The new
rate limit is configured so that it takes 24 hours to eat 10 gigabytes:
10 * 1024 / 8 / (24 * 60 / 30) = 27 (which rounded to 30)

The period is set long so that it does not prevent from attaching several
media to one post, which would happen in a short period. For example,
if the period is 5 minutes, the rate limit would be:
10 * 1024 / 8 / (24 * 60 / 5) = 4

This long period allows to lift the limit up.
2018-05-03 17:32:00 +02:00
Eugen Rochko ca1c696dbd Slightly reduce RAM usage (#7301)
* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts
2018-05-02 18:58:48 +02:00
MIYAGI Hikaru 28808f638e HTTP proxy support for outgoing request, manage access to hidden service (#7134)
* Add support for HTTP client proxy

* Add access control for darknet

Supress error when access to darknet via transparent proxy

* Fix the codes pointed out

* Lint

* Fix an omission + lint

* any? -> include?

* Change detection method to regexp to avoid test fail
2018-04-25 02:14:49 +02:00
Yamagishi Kazutoshi 9761b940ac Upgrade Rails to version 5.2.0 (#5898) 2018-04-12 14:45:17 +02:00
Eugen Rochko dbf7f62cea Use RAILS_LOG_LEVEL to set log level of Sidekiq, too (#7079)
Fix #3565 (oops)
2018-04-10 16:08:28 +02:00
Eugen Rochko 8af5b4ed4b Log rate limit hits (#7096)
Fix #7095
2018-04-10 01:20:18 +02:00
Eugen Rochko a814aa8e4d Add a circuit breaker for ActivityPub deliveries (#7053) 2018-04-07 21:36:58 +02:00
Yamagishi Kazutoshi 6daa722e87 Revert "Revert "Upgrade Paperclip to version 6.0.0" (#6807)" (#6808)
This reverts commit d35272245e.
2018-03-24 12:52:45 +01:00
Eugen Rochko 59b3b38b0e Add LDAP_TLS_NO_VERIFY option, don't require LDAP_ENABLED outside .env (#6845)
Fix #6816, fix #6790
2018-03-20 19:41:51 +01:00
Alexander af08f6042d rename pam email environment variable to something more understandable and default to LOCAL_DOMAIN (better fallback) (#6833) 2018-03-19 20:09:26 +01:00
Eugen Rochko d35272245e Revert "Upgrade Paperclip to version 6.0.0" (#6807)
* Revert "Bump version to 2.3.2rc1"

This reverts commit 64d5c8a512.

* Revert "Downgrade Dockerfile to Ruby 2.4.3 on Alpine 3.6 (#6806)"

This reverts commit 36734278ba.

* Revert "Handle Mastodon::HostValidationError when pulling remoteable assets (#6782)"

This reverts commit 8f374100ed.

* Revert "Correct the reference to user's password in mastodon:add_user task (#6800)"

This reverts commit 7b247b15f2.

* Revert "Upgrade Paperclip to version 6.0.0 (#6754)"

This reverts commit cd6dee83a1.
2018-03-17 14:20:35 +01:00
Yamagishi Kazutoshi cd6dee83a1 Upgrade Paperclip to version 6.0.0 (#6754) 2018-03-17 12:37:58 +01:00
Effy Elden 4a8046df66 Add additional first_name and last_name SAML attribute statement options, and modify Omniauthable concern to use full_name or first_name + last_name if not available (#6669) 2018-03-07 06:19:10 +01:00
Alexander 988f6505e4 fix logic for pam_controlled_service (#6599) 2018-03-02 19:02:50 +01:00