Commit Graph

127 Commits (1c28a202feca51e652d0ddb9e7777d6ba2547414)

Author SHA1 Message Date
Nick Schonning 2f2c8f671b Autofix Rubocop Style/HashSyntax (#23754) 2023-05-04 05:54:26 +02:00
Matt Jankowski e8537a3a01 Fix Performance/CollectionLiteralInLoop cop (#24819) 2023-05-04 05:33:55 +02:00
Matt Jankowski ec265b7d8a Fix Rails/ActiveRecordCallbacksOrder cop (#24689) 2023-05-02 18:23:35 +02:00
Eugen Rochko 1d2a941bc8 Change media upload limits and remove client-side resizing (#23726) 2023-03-25 10:00:03 +01:00
Nick Schonning e6312104c7 Autofix Rubocop remaining Layout rules (#23679) 2023-02-20 06:58:28 +01:00
Nick Schonning 448f0929a7 Autofix Rubocop Rails/EnumHash (#23737) 2023-02-20 05:00:36 +01:00
Claire b821678966 Change remote media files to be downloaded outside of transactions (#21796) 2022-12-15 18:09:48 +01:00
Skyler Hawthorne ce10b313bb fix media uploads with ffmpeg 5 (#21191) 2022-11-25 16:20:47 +01:00
Eugen Rochko 2018fe879c Fix image type not being set after conversion for convertible image types (#20624) 2022-11-14 07:13:14 +01:00
Eugen Rochko 361d6793e8 Fix metadata scrubbing removing color profile from images (#20389) 2022-11-11 09:20:10 +01:00
txt-file a2703b8852 Add support for AVIF uploads (#19647) 2022-11-01 22:08:41 +01:00
Eugen Rochko ca1dec3831 Add support for HEIC uploads (#19618) 2022-11-01 16:26:25 +01:00
Eugen Rochko 529fe25add Change post-processing to be deferred only for large media types (#19617) 2022-11-01 15:27:58 +01:00
Eugen Rochko 0c0daf7142 Change max. thumbnail dimensions to 640x360px (360p) (#19619) 2022-11-01 13:01:39 +01:00
Jeong Arm 386980c196 Support audio/vnd.wave (#18737)
See: https://datatracker.ietf.org/doc/html/rfc2361
And Misskey uses this mime type for wav file.
2022-06-28 19:49:35 +02:00
Alexander Ivanov a59e8245f5 Add support for webp uploads (#18506) 2022-05-27 20:06:40 +02:00
Eugen Rochko 88f93d03fe Fix edits with no actual changes being allowed (#17843)
* Fix edits with no actual changes being allowed locally

* Fix edits with no actual changes being allowed through ActivityPub

* Fix false positive changes caused by description processing in model

* Fix not recording poll expiration update

* Fix test

* Revert changes to ProcessStatusUpdateService

* Various fixes and improvements

* Fix code style issues

* Various changes and improvements

* Add guard clause
2022-03-26 00:38:44 +01:00
Claire f499f086a9 Fix some media attachments being converted with too high framerates (#17619)
Video files with variable framerates are converted to constant framerate videos
and the output framerate picked by ffmpeg is based on the original file's
container framerate (which can be different from the average framerate).

This means that an input video with variable framerate with about 30 frames per
second on average, but a maximum of 120 fps will be converted to a constant 120
fps file, which won't be processed by other Mastodon servers.

This commit changes it so that input files with VFR and a maximum framerate
above the framerate threshold are converted to VFR files with the maximum frame
rate enforced.
2022-02-22 17:11:22 +01:00
Eugen Rochko cb76142d9e Add editing for published statuses (#17320)
* Add editing for published statuses

* Fix change of multiple-choice boolean in poll not resetting votes

* Remove the ability to update existing media attachments for now
2022-02-10 00:15:30 +01:00
Claire 879229e098 Fix error when rendering public pages with media attachments (#16763)
* Add tests

* Fix error when rendering public pages with media attachments

* Add tests

* Fix tests

* Please CodeClimate
2021-10-13 15:27:19 +02:00
Claire 10918d97d4 Fix media attachment size validation not correctly accounting for file type (#16819)
* Fix media attachment size validation not correctly accounting for file type

Fixes a regression introduced in #16724 caused by the fact that kt-paperclip
now correctly runs validations before processing, meaning that file size
verification could not rely on our before_post_processing hook.

Moved the `before_post_processing` hooks to `before_validate` to make sure
the media attachment type is set correctly before the file gets validated.

* Add tests
2021-10-06 14:49:32 +01:00
Claire 7d0089033f Switch from unmaintained paperclip to kt-paperclip (#16724)
* Switch from unmaintained paperclip to kt-paperclip

* Drop some compatibility monkey-patches not required by kt-paperclip

* Drop media spoof check monkey-patching

It's broken with kt-paperclip and hopefully it won't be needed anymore

* Fix regression introduced by paperclip 6.1.0

* Do not rely on pathname to call FastImage

* Add test for ogg vorbis file with cover art

* Add audio/vorbis to the accepted content-types

This seems erroneous as this would be the content-type for a vorbis stream
without an ogg container, but that's what the `marcel` gem outputs, so…

* Restore missing for_as_default method

* Refactor Attachmentable concern and delay Paperclip's content-type spoof check

Check for content-type spoofing *after* setting the extension ourselves, this
fixes a regression with kt-paperclip's validations being more strict than
paperclip 6.0.0 and rejecting some Pleroma uploads because of unknown
extensions.

* Please CodeClimate

* Add audio/vorbis to the unreliable set

It doesn't correspond to a file format and thus has no extension associated.
2021-09-29 23:52:36 +02:00
Claire 4080a8ec33 Stop setting a shortcode to newly-created media attachments (#16730)
* Stop setting a shortcode to newly-created media attachments

The WebUI has stopped using the “short media URL” in ages. This isn't used
anywhere except for mail notifications.

Deprecating it would allow us to eventually get rid of at least a database
column and corruption-prone index, as well as a controller.

* Fix tests
2021-09-13 18:59:37 +02:00
Eugen Rochko 1c2b085a64 Fix media processing getting stuck on too much stdin/stderr (#16136)
* Fix media processing getting stuck on too much stdin/stderr

See thoughtbot/terrapin#5

* Remove dependency on paperclip-av-transcoder gem

* Remove dependency on streamio-ffmpeg gem

* Disable stdin on ffmpeg process
2021-05-05 19:44:01 +02:00
Eugen Rochko 826a5cc624 Change max. image dimensions to 1920x1080px (1080p) (#15690)
* Change max. image size to 1920x1080px

* Change it in web UI too
2021-02-09 01:21:06 +01:00
Eugen Rochko 97762d28b6 Fix videos with near-60 fps being rejected (#14684)
Fix #14668
2020-08-30 01:54:30 +02:00
Takeshi Umeda 622dcbeb8c Fix movie width and frame_rate returning nil (#14357)
* Fix movie width and frame_rate returning nil

* Add StreamValidationError and raise

* Fix code style
2020-07-19 22:28:27 +02:00
ThibG d93a24936c Fix larger video files not being transcoded (#14306)
Since #14145, the `set_type_and_extension` has been moved from
`before_post_process` to `before_file_post_process`, but while the former
runs before all validations performed by Paperclip, the latter is dependent
on the order validations and hooks are defined.

In our case, this meant video files could be checked against the generic 10MB
limit, causing validation failures, which, internally, make Paperclip skip
post-processing, and thus, transcoding of the video file.

The actual validation would then happen after the type is correctly set, so
the large file would pass validation, but without being transcoded first.

This commit moves the hook definition so that it is run before checking for
the file size.
2020-07-14 18:50:19 +02:00
Eugen Rochko 8517a5fdb4 Add color extraction for audio thumbnails (#14209) 2020-07-05 18:28:25 +02:00
Eugen Rochko a3ce01a102 Add customizable thumbnails for audio and video attachments (#14145)
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
2020-06-29 13:56:55 +02:00
Eugen Rochko 12ac904a33 Fix various issues around OpenGraph representation of media (#14133)
- Fix audio attachments not being represented in OpenGraph tags
- Fix audio being represented as "1 image" in OpenGraph descriptions
- Fix video metadata being overwritten by paperclip-av-transcoder
- Fix embedded player not using Mastodon's UI
- Fix audio/video progress bars not moving smoothly
- Fix audio/video buffered bars not displaying correctly
2020-06-25 01:33:01 +02:00
Eugen Rochko adfb22a9dd Add separate cache directory for non-local uploads (#12821) 2020-04-26 23:29:08 +02:00
Eugen Rochko 1cb4645638 Change `tootctl media remove-orphans` to work for all classes (#13316)
Change `tootctl media lookup` to not use an interactive prompt
2020-03-26 01:56:41 +01:00
ThibG 0701e09c85 Fix videos with unsupported colorspace not being transcoded (#13242) 2020-03-10 11:58:40 +01:00
ThibG ccd8ceac47 Fix MP4 (H264 + AAC) video files being needlessly re-encoded (#13239) 2020-03-09 23:15:59 +01:00
Eugen Rochko a0c60d09ff Change video uploads to always be converted to H264/MP4 (#13220)
Even if the container format is the same (.mp4), the codec could
be different and not playable in web browsers
2020-03-09 02:20:18 +01:00
Eugen Rochko 0995464585 Change video uploads to enforce certain limits (#13218)
- Dimensions at most 1920x1200
- Frame rate at most 60
2020-03-09 02:19:07 +01:00
Eugen Rochko f459919552 Change local media attachments to perform heavy processing asynchronously (#13210)
Fix #9106
2020-03-08 23:56:18 +01:00
Eugen Rochko df086209a2 Fix "tootctl media remove-orphans" crashing when encountering invalid media (#13170)
Fixes #13168
2020-03-05 15:56:01 +01:00
Eugen Rochko 930ba0630f Fix media attachments without file being uploadable (#12562)
Fix #12554
2020-01-23 21:40:03 +01:00
Eugen Rochko 413cb7e861 Fix base64-encoded file uploads not being possible (#12748)
Fix #3804, Fix #5776
2020-01-04 01:54:07 +01:00
Eugen Rochko 0769ef8277 Add `tootctl media remove-orphans` (#12568) 2019-12-08 15:37:12 +01:00
Yamagishi Kazutoshi b01cb4701e Update ESLint and RuboCop in Code Climate (#12534) 2019-12-02 18:25:43 +01:00
ThibG 479d8ff47a Fix remote media descriptions being cut off at 420 chars (#12262)
* Fix remote media descriptions being cut off at 420 chars

Fixes #12258

* Fix tests
2019-11-04 13:00:16 +01:00
Hugo Gameiro 9980046428 remove audio metadata (#12171) 2019-10-24 22:47:58 +02:00
Hugo Gameiro b302921eb1 add loglevel fatal to video and audio styles (#12088) 2019-10-06 19:48:26 +02:00
Eugen Rochko 746fae0bd1 Fix performance of GIF re-encoding (#12057)
* Change animated GIF detection to not shell out to ImageMagick

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Change video encoding parameters to limit to 10800 video frames

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Limit GIF image size further

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Always strip metadata from video files

* Fix code style issues
2019-10-03 01:09:12 +02:00
Eugen Rochko 9301574423 Change max length of media descriptions from 420 to 1500 characters (#11819)
Fix #11658
2019-09-13 16:00:34 +02:00
Eugen Rochko 8eab4349b4 Add retry for failed media downloads and `tootctl media refresh` (#11775) 2019-09-10 15:29:12 +02:00
Eugen Rochko 666c1a8a06 Change tootctl to use inline parallelization instead of Sidekiq (#11776)
- Remove --background option
- Add --concurrency(=5) option
- Add progress bars
2019-09-10 13:48:48 +02:00