Commit Graph

238 Commits (d01cb2c8e36da70ca90475b2c7b983309a7e8002)

Author SHA1 Message Date
David Yip ea46024add Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
 	Gemfile.lock
2018-05-04 09:42:31 -05:00
Eugen Rochko 860701a9a8 Marginally improve convert/ffmpeg calls performance with posix-spawn (#7346) 2018-05-04 13:22:23 +02:00
David Yip c87f1d99e4 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	.travis.yml
 	Gemfile.lock
 	README.md
 	app/controllers/settings/follower_domains_controller.rb
 	app/controllers/statuses_controller.rb
 	app/javascript/mastodon/locales/ja.json
 	app/lib/feed_manager.rb
 	app/models/media_attachment.rb
 	app/models/mute.rb
 	app/models/status.rb
 	app/services/mute_service.rb
 	app/views/home/index.html.haml
 	app/views/stream_entries/_simple_status.html.haml
 	config/locales/ca.yml
 	config/locales/en.yml
 	config/locales/es.yml
 	config/locales/fr.yml
 	config/locales/nl.yml
 	config/locales/pl.yml
 	config/locales/pt-BR.yml
 	config/themes.yml
2018-05-03 17:23:44 -05:00
Shuhei Kitagawa 7236e9effe Add pry-byebug (#7307) 2018-05-03 12:08:54 +02:00
Eugen Rochko ca1c696dbd Slightly reduce RAM usage (#7301)
* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts
2018-05-02 18:58:48 +02:00
Yamagishi Kazutoshi e83e74f36d Update dependencies for Ruby (2018-04-23) (#7237)
* Update annotate to version 2.7.3

* Update aws-sdk-s3 to version 1.9.2

* Update browser to version 2.5.3

* Update capistrano to version 3.10.2

* Update domain_name to version 0.5.20180417

* Update http to version 3.2.0

* Update lograge to version 0.10.0

* Update oj to version 3.5.1

* Update parallel_tests to version 2.21.3

* Update puma to version 3.11.4

* Update rubocop to version 0.55.0

* Update scss_lint to version 0.57.0

* Update simplecov to version 0.16.1

* Update tty-command to version 0.8.0

* Update tty-prompt to version 0.16.0

* Update pkg-config to version 1.3.0

* Update fog-local to version 0.5.0

* Update fog-openstack to version 0.1.25

* Update devise-two-factor to version 3.0.3

* bundle update
2018-04-23 11:29:17 +02:00
Yamagishi Kazutoshi 0d4342e146 Introduce rspec-retry (#7206) 2018-04-20 19:31:30 +02:00
David Yip fd98bfd108 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	Gemfile.lock
 	config/application.rb
2018-04-13 16:36:46 -05:00
Yamagishi Kazutoshi 9761b940ac Upgrade Rails to version 5.2.0 (#5898) 2018-04-12 14:45:17 +02:00
David Yip ac1ac20ee9 Merge remote-tracking branch 'tootsuite/master'
Conflicts:
 	app/controllers/statuses_controller.rb
2018-04-12 03:30:57 -05:00
Alexander 8a9da4c414 update gem, test pam authentication (#7028)
* update gem, test pam authentication

* add description for test parameters

* fix inclusion of optional group
2018-04-11 21:40:38 +02:00
David Yip 337fc136fd Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/serializers/initial_state_serializer.rb

The glitch flavour isn't yet pulling custom emoji data on its own (see
https://github.com/tootsuite/mastodon/pull/7047).  Once that gets into
the glitch flavour, we can eliminate the custom_emojis load.
2018-04-08 19:05:02 -05:00
Eugen Rochko a814aa8e4d Add a circuit breaker for ActivityPub deliveries (#7053) 2018-04-07 21:36:58 +02:00
Jenkins 658440838b Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-30 19:17:17 +00:00
Pierre-Morgan Gate ccacb263da Upgrade charlock_holmes to version 0.7.6 (#6966)
This version fixes compilation errors when trying to build its native extension with ICU 61.
2018-03-30 09:52:44 +02:00
Jenkins ae55717f50 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-25 15:17:21 +00:00
Yamagishi Kazutoshi 6daa722e87 Revert "Revert "Upgrade Paperclip to version 6.0.0" (#6807)" (#6808)
This reverts commit d35272245e.
2018-03-24 12:52:45 +01:00
Jenkins a9bcd383b8 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-21 17:17:20 +00:00
Patrick Figel e62904bcba Update sanitize and loofah (#6855)
Fixes CVE-2018-8048 and CVE-2018-3740, two medium-severity XSS
vulnerabilities present in these gems when built against
libxml2 >= 2.9.2.
2018-03-21 17:43:28 +01:00
Jenkins eba1b109db Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-19 20:17:18 +00:00
Alexander 4fc9827b69 bugfixes and gem update (#6831)
* update to new version of devise_pam_authenticatable2

* fix behaviour if suffix is nil, fix environment loading, fix user email creation

* code cleanup/fix linter warning
2018-03-19 20:08:56 +01:00
Eugen Rochko d35272245e Revert "Upgrade Paperclip to version 6.0.0" (#6807)
* Revert "Bump version to 2.3.2rc1"

This reverts commit 64d5c8a512.

* Revert "Downgrade Dockerfile to Ruby 2.4.3 on Alpine 3.6 (#6806)"

This reverts commit 36734278ba.

* Revert "Handle Mastodon::HostValidationError when pulling remoteable assets (#6782)"

This reverts commit 8f374100ed.

* Revert "Correct the reference to user's password in mastodon:add_user task (#6800)"

This reverts commit 7b247b15f2.

* Revert "Upgrade Paperclip to version 6.0.0 (#6754)"

This reverts commit cd6dee83a1.
2018-03-17 14:20:35 +01:00
Yamagishi Kazutoshi cd6dee83a1 Upgrade Paperclip to version 6.0.0 (#6754) 2018-03-17 12:37:58 +01:00
Jenkins 54608d0486 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-09 00:17:17 +00:00
Yamagishi Kazutoshi 3008a0f73a Upgrade chewy to version 5.0.0 (#6649) 2018-03-06 06:14:26 +01:00
David Yip c08c971dd3 Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
 	README.md
 	app/controllers/follower_accounts_controller.rb
 	app/controllers/following_accounts_controller.rb
 	app/serializers/rest/instance_serializer.rb
 	app/views/stream_entries/_simple_status.html.haml
 	config/locales/simple_form.ja.yml
2018-03-02 21:46:44 -06:00
Patrick Figel faa3fe644d Update omniauth-saml to 1.10 (#6587)
Fixes CVE-2017-11428
2018-03-02 02:32:08 +01:00
Eugen Rochko 9721b7746a Fix #942: Seamless LDAP login (#6556) 2018-02-28 19:04:53 +01:00
masarakki eed7c16ab0 remove-uglifier (#6561) 2018-02-26 16:19:48 +01:00
Akihiko Odaki a5a434a8f6 Raise Mastodon::HostValidationError when host for HTTP request is private (#6410) 2018-02-24 19:16:11 +01:00
imncls c0aabbec0f Merge branch 'master' of https://github.com/tootsuite/mastodon
# Conflicts:
#	app/controllers/settings/exports_controller.rb
#	app/models/media_attachment.rb
#	app/models/status.rb
#	app/views/about/show.html.haml
#	docker_entrypoint.sh
#	spec/views/about/show.html.haml_spec.rb
2018-02-23 23:28:31 +09:00
Eugen Rochko f6884555d7 Account archive download (#6460)
* Fix #201: Account archive download

* Export actor and private key in the archive

* Optimize BackupService

- Add conversation to cached associations of status, because
  somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
  (Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
  (Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
  (Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
  GIF-to-MP4 media are all named "media.mp4" for some reason

* Keep uniquely generated filename in Paperclip::GifTranscoder

* Ensure dumped files do not overwrite each other by maintaing directory partitions

* Give tar archives a good name

* Add scheduler to remove week-old backups

* Fix code style issue
2018-02-21 23:21:32 +01:00
Eugen Rochko c34ff9fb82 Fix avatar and header issues by using custom geometry detector (#6515)
* Fix avatar and header issues by using custom geometry detector

Revert a part of #6508. The file passed to dynamic styles method
was not actually a file, but an instance of Paperclip::Attachment,
which broke all styles by always returning {} from the method.

One problem with GIF avatars was that Paperclip::GeometryDetector
reported wrong dimensions for them, e.g. 120x120 GIF avatar would
for some reason be detected as 120x53. By writing our own geometry
parser, we can use FastImage, which also happens to be faster than
ImageMagick, to detect image dimensions, which are also correct.

Unfortunately, this PR does not implement skipping a `convert`
entirely if the dimensions are already correct, as I found no easy
way to write that behaviour into Paperclip without rewriting the
Paperclip::Thumbnail class.

* Only invoke convert if dimension or format needs to be changed
2018-02-21 03:40:12 +01:00
David Yip 596deb050b Merge remote-tracking branch 'tootsuite/master' into merge-upstream
Conflicts:
	Gemfile
	config/locales/simple_form.pl.yml
2018-02-17 00:02:37 -06:00
Eugen Rochko 9b7490cede Save video metadata and improve video OpenGraph tags (#6481)
* Save metadata from video attachments, put correct dimensions into OG tags

* Add twitter:player for videos

* Fix code style and test
2018-02-16 07:22:20 +01:00
Jenkins bf93764ca0 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-11 18:17:11 +00:00
Eugen Rochko e8e6786d6f Interactive `rake mastodon:setup` task (#6451)
* Add better CLI prompt

* Add rake mastodon:setup interactive wizard

* Test db/redis/smtp configurations and add admin user at the end

* Test database connection even when database does not exist yet
2018-02-11 18:40:57 +01:00
Jenkins 3a1f58e9eb Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-11 04:10:16 +00:00
David Yip 79f594fcba Introduce html2text for extracting plaintext from statuses. #236.
Unlike strip_tags, html2text will preserve text present in other nodes,
e.g. anchor tags:

    [1] pry(main)> str = '<a href="http://www.example.com">A link</a>'
    => "<a href=\"http://www.example.com\">A link</a>"
    [2] pry(main)> Html2Text.convert(str)
    => "[A link](http://www.example.com)"
    [3] pry(main)> include ActionView::Helpers::SanitizeHelper
    => Object
    [4] pry(main)> strip_tags(str)
    => "A link"

Preserving the href of an anchor allows keyword mutes to also match on
URLs, which is something that the frontend regex filter can currently
do.
2018-02-10 10:36:16 -06:00
Eugen Rochko 7ca5a06505 Full-text search for authorized statuses (#6423)
* Add full-text search for authorized statuses

- Search API will return statuses that match the query
- Only for logged in users
- Only if you are author of the status,
- Or you were mentioned in it
- Or you favourited or reblogged it
- Configuration over `ES_ENABLED`, `ES_HOST`, `ES_PORT`, `ES_PREFIX`
- Run `rails chewy:deploy` to create & populate index

Fix #5880
Fix #4293
Fix #1152

* Add commented out docker-compose configuration for ES container

* Optimize index import, filter search results

* Add basic normalization to the index

* Add better stemming and normalization to the index

* Skip webfinger request if search query includes both @ and a space

* Fix code style

* Visually separate search result sections

* Fix code style issues
2018-02-09 23:04:47 +01:00
David Yip 4f8122a98c Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	.env.production.sample
	app/controllers/auth/confirmations_controller.rb
	db/schema.rb
2018-02-04 16:36:19 -06:00
Eugen Rochko 5322013f25 CAS + SAML authentication feature (#6425)
* Cas authentication feature

* Config

* Remove class_eval + Omniauth initializer

* Codeclimate review

* Codeclimate review 2

* Codeclimate review 3

* Remove uid/email reconciliation

* SAML authentication

* Clean up code

* Improve login form

* Fix code style issues

* Add locales
2018-02-04 05:42:13 +01:00
David Yip 6d1023b2e9 Merge remote-tracking branch 'tootsuite/master' into merge-upstream
Conflicts:
      app/javascript/styles/mastodon/components.scss
2018-02-02 08:39:52 -06:00
Alexander 23ce0c86da pam authentication (#5303)
* add pam support, without extra column

* bugfixes for pam login

* document options

* fix code style

* fix codestyle

* fix tests

* don't call remember_me without password

* fix codestyle

* improve checks for pam usage (should fix tests)

* fix remember_me part 1

* add remember_token column because :rememberable requires either a password or this column.

* migrate db for remember_token

* move pam_authentication to the right place, fix logic bug in edit.html.haml

* fix tests

* fix pam authentication, improve username lookup, add comment

* valid? is sometimes not honored, return nil instead trying to authenticate with pam

* update devise_pam_authenticatable2 and adjust code. Fixes sideeffects observed in tests

* update devise_pam_authenticatable gem, fixes for codeconventions, fix finding user

* codeconvention fixes

* code convention fixes

* fix idention

* update dependency, explicit conflict check

* fix disabled password updates if in pam mode

* fix check password if password is present, fix templates

* block registration if account is maintained by pam

* Revert "block registration if account is maintained by pam"

This reverts commit 8e7a083d650240b6fac414926744b4b90b435f20.

* fix identation error introduced by rebase

* block usernames maintained by pam

* document pam settings better

* fix code style
2018-02-02 10:18:55 +01:00
Rob Watson 564e2c448b Upgrade Paperclip > 5.2.1 (#6404)
Mitigation for CVE-2017-0889.

https://www.cvedetails.com/cve/CVE-2017-0889/
https://medium.com/in-the-weeds/all-about-paperclips-cve-2017-0889-server-side-request-forgery-ssrf-vulnerability-8cb2b1c96fe8
2018-02-01 17:54:22 +01:00
Jenkins 3cd4a0804b Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-24 19:17:11 +00:00
Eugen Rochko 0efaa528c4 Update goldfinger, ostatus2 and http.rb versions (#6337) 2018-01-23 14:30:42 +01:00
David Yip a95ca95b7b Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	app/controllers/settings/two_factor_authentication/confirmations_controller.rb
2018-01-21 13:36:10 -06:00
Aboobacker MK a1979b50c9 Redirect to 2FA creation page when otp_secret is not available (#6314) 2018-01-21 13:21:28 +01:00
David Yip d45ebe26dc Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	app/javascript/styles/mastodon/components.scss
	app/javascript/styles/mastodon/modal.scss
2018-01-19 15:22:10 -06:00