Commit Graph

84 Commits (58e52929be80cdd0d2910595097fa3e89113b0c5)

Author SHA1 Message Date
ThibG 479d8ff47a Fix remote media descriptions being cut off at 420 chars (#12262)
* Fix remote media descriptions being cut off at 420 chars

Fixes #12258

* Fix tests
2019-11-04 13:00:16 +01:00
Hugo Gameiro 9980046428 remove audio metadata (#12171) 2019-10-24 22:47:58 +02:00
Hugo Gameiro b302921eb1 add loglevel fatal to video and audio styles (#12088) 2019-10-06 19:48:26 +02:00
Eugen Rochko 746fae0bd1 Fix performance of GIF re-encoding (#12057)
* Change animated GIF detection to not shell out to ImageMagick

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Change video encoding parameters to limit to 10800 video frames

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Limit GIF image size further

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Always strip metadata from video files

* Fix code style issues
2019-10-03 01:09:12 +02:00
Eugen Rochko 9301574423 Change max length of media descriptions from 420 to 1500 characters (#11819)
Fix #11658
2019-09-13 16:00:34 +02:00
Eugen Rochko 8eab4349b4 Add retry for failed media downloads and `tootctl media refresh` (#11775) 2019-09-10 15:29:12 +02:00
Eugen Rochko 666c1a8a06 Change tootctl to use inline parallelization instead of Sidekiq (#11776)
- Remove --background option
- Add --concurrency(=5) option
- Add progress bars
2019-09-10 13:48:48 +02:00
umonaca 70e0c2f162 Add more media formats (#11687)
* Add more media formats

* Removed amr file type
2019-08-30 04:30:29 +02:00
Stanislas 8f7051ee33 Remove WebP support (#11589) 2019-08-17 22:04:15 +02:00
Eugen Rochko 2854c8e4d6 Remove timestamps from converted images to make them deterministic (#11408) 2019-07-27 04:41:55 +02:00
Eugen Rochko 01cff5ee60 Add aac, m4a, 3gp to allowed audio formats (#11342)
Fix #11186
2019-07-18 03:02:30 +02:00
Eugen Rochko 297e15baa0 Fix audio-only OGG and WebM files not being processed as such (#11151)
Also, because Chrome sends audio/mp3 instead of audio/mpeg as it's
supposed to, we need to whitelist that mime type as well
2019-06-22 16:54:06 +02:00
Eugen Rochko d85b2a734b Fix audio not being downloaded from remote servers (#11145) 2019-06-22 02:50:36 +02:00
Eugen Rochko a266d7f1a2 Change audio format from ogg to mp3 for wider compatibility (#11141)
* Change audio format from ogg to mp3 for wider compatibility

* Add media description as title to links of unknown media attachments
2019-06-21 22:59:44 +02:00
Eugen Rochko c078bf94a9 Fix converted media being saved with original extension and mime type (#11130) 2019-06-20 10:52:36 +02:00
Eugen Rochko d7e2f554fb Add audio uploads (#11123)
* Add audio uploads

Fix #4827

Accept uploads of OGG, WAV, FLAC, OPUS and MP3 files, and converts
them to OGG. Media attachments get a new `audio` type. In the UI,
audio uploads are displayed identically to video uploads.

* Improve code style
2019-06-19 23:42:38 +02:00
Eugen Rochko a6d2fe7165 Add blurhash (#10630)
* Add blurhash

* Use fallback color for spoiler when blurhash missing

* Federate the blurhash and accept it as long as it's at most 5x5

* Display unknown media attachments as blurhash placeholders

* Improve style of embed actions and spoiler button

* Change blurhash resolution from 3x3 to 4x4

* Improve dependency definitions

* Fix code style issues
2019-04-27 03:24:09 +02:00
rinsuki 18af20b9cf Use video filesize limit with gifv (#9924) 2019-02-04 04:46:05 +01:00
Acid Chicken (硫酸鶏) bcf34e31bd Add WebP support (#9879)
* Add WebP support

* Remove the changes to the tooltip

refs: https://github.com/tootsuite/mastodon/pull/9879#pullrequestreview-199312528
2019-02-02 12:25:04 +01:00
Eugen Rochko ae1aaa3b8a Add scheduled statuses (#9706)
Fix #340
2019-01-05 12:43:28 +01:00
Hugo Gameiro 92238e2f7a add loglevel to ffmpeg in gif upload (#9368) 2018-11-27 12:19:12 +01:00
Eugen Rochko c3005d09e6 Reset status cache when status_stat or media_attachment updates (#9119)
* Reset status cache when status_stat or media_attachment updates

Fix #8711

Media attachments are generally immutable, but admins can update
the sensitive flag, and this would ensure the change is visible
instantly. Same for updates to status stats. That is a regression
from #8185, because even the correct updated_at fetched from a join
doesn't seem to invalidate the cache.

* Remove join from Status#cache_ids since it has no effect
2018-10-28 06:42:34 +01:00
Eugen Rochko f7910c4164 Convert MOV and WEBM to MP4, raise maximum limit to 40MB (#8101)
Separate size limits for images and video. Images remain at 8MB,
while videos can be up to 40MB.
2018-07-30 19:33:05 +02:00
Eugen Rochko 91aadec929 Resize images by area instead of fixed dimensions (#8083)
To improve the way super tall or super ride images are treated, the
numbers remain the same, 1280x1280 and 400x400, but if an image
is less in one dimension than the other, the other can become larger

Thanks to @WAHa_06x36@mastodon.social for the tip
2018-07-28 03:33:00 +02:00
Yamagishi Kazutoshi e83e74f36d Update dependencies for Ruby (2018-04-23) (#7237)
* Update annotate to version 2.7.3

* Update aws-sdk-s3 to version 1.9.2

* Update browser to version 2.5.3

* Update capistrano to version 3.10.2

* Update domain_name to version 0.5.20180417

* Update http to version 3.2.0

* Update lograge to version 0.10.0

* Update oj to version 3.5.1

* Update parallel_tests to version 2.21.3

* Update puma to version 3.11.4

* Update rubocop to version 0.55.0

* Update scss_lint to version 0.57.0

* Update simplecov to version 0.16.1

* Update tty-command to version 0.8.0

* Update tty-prompt to version 0.16.0

* Update pkg-config to version 1.3.0

* Update fog-local to version 0.5.0

* Update fog-openstack to version 0.1.25

* Update devise-two-factor to version 3.0.3

* bundle update
2018-04-23 11:29:17 +02:00
Eugen Rochko 39f36c0901 Detect and prevent image bombs, max. processable dimension 4096^2 (#7229) 2018-04-23 09:16:38 +02:00
MIYAGI Hikaru 62c6b71866 Fix GIFV encoding params (#7098)
- Explicitly specify video codec.
  When ffmpeg isn't compiled with libx264 but openh264, mpeg4 is selected as video codec.
- Swap avarage bitrate and max bitrate.
2018-04-10 09:11:55 +02:00
Akihiko Odaki acece7a2e6 Validate HTTP response length while receiving (#6891)
to_s method of HTTP::Response keeps blocking while it receives the whole
content, no matter how it is big. This means it may waste time to receive
unacceptably large files. It may also consume memory and disk in the
process. This solves the inefficency by checking response length while
receiving.
2018-03-26 14:02:10 +02:00
Eugen Rochko 865c7e7178 Focal points (#6520)
* Add focus param to media API, center thumbnails on focus point

* Add UI for setting a focal point

* Improve focal point icon on upload item

* Use focal point in upload preview

* Add focalPoint property to ActivityPub

* Don't show focal point button for non-image attachments
2018-02-22 00:35:46 +01:00
Eugen Rochko c34ff9fb82 Fix avatar and header issues by using custom geometry detector (#6515)
* Fix avatar and header issues by using custom geometry detector

Revert a part of #6508. The file passed to dynamic styles method
was not actually a file, but an instance of Paperclip::Attachment,
which broke all styles by always returning {} from the method.

One problem with GIF avatars was that Paperclip::GeometryDetector
reported wrong dimensions for them, e.g. 120x120 GIF avatar would
for some reason be detected as 120x53. By writing our own geometry
parser, we can use FastImage, which also happens to be faster than
ImageMagick, to detect image dimensions, which are also correct.

Unfortunately, this PR does not implement skipping a `convert`
entirely if the dimensions are already correct, as I found no easy
way to write that behaviour into Paperclip without rewriting the
Paperclip::Thumbnail class.

* Only invoke convert if dimension or format needs to be changed
2018-02-21 03:40:12 +01:00
Eugen Rochko 9b7490cede Save video metadata and improve video OpenGraph tags (#6481)
* Save metadata from video attachments, put correct dimensions into OG tags

* Add twitter:player for videos

* Fix code style and test
2018-02-16 07:22:20 +01:00
Yamagishi Kazutoshi 49b76b3025 Change belongs_to_required_by_default to true (#5888) 2018-01-19 20:56:47 +01:00
Eugen Rochko e149067561 Lists (#5703)
* Add structure for lists

* Add list timeline streaming API

* Add list APIs, bind list-account relation to follow relation

* Add API for adding/removing accounts from lists

* Add pagination to lists API

* Add pagination to list accounts API

* Adjust scopes for new APIs

- Creating and modifying lists merely requires "write" scope
- Fetching information about lists merely requires "read" scope

* Add test for wrong user context on list timeline

* Clean up tests
2017-11-18 00:16:48 +01:00
Daniel Hunsaker 96504fd1ec Update model annotations to use BIGINT for IDs (#5461)
All the migrations have been updated to use BIGINTs for ID fields in the DB, but ActiveRecord needs to be told to treat those values as BIGINT as well. This PR does that.
2017-11-12 16:18:50 +01:00
Eugen Rochko 5f137e7ec3 Increase attachment descriptions to 420 characters (#5139)
Blaze it
2017-09-29 02:30:00 +02:00
Eugen Rochko 795e624d5d Add ability to specify alternative text for media attachments (#5123)
* Fix #117 - Add ability to specify alternative text for media attachments

- POST /api/v1/media accepts `description` straight away
- PUT /api/v1/media/:id to update `description` (only for unattached ones)
- Serialized as `name` of Document object in ActivityPub
- Uploads form adjusted for better performance and description input

* Add tests

* Change undo button blend mode to difference
2017-09-28 15:31:31 +02:00
unarist ca502e7b15 Use file extensions in addition to MIME types for file picker (#5029)
Currently we're using a list of MIME types for `accept` attribute on `input[type="file"]` for filter options of file picker, and actual file extensions will be infered by browsers. However, infered extensions may not include our expected items. For example, "image/jpeg" seems to be infered to
only ".jfif" extension in Firefox.

To ensure common file extensions are in the list, this PR adds file extensions in addition to MIME types. Also having items in both format is encouraged by HTML5 spec.

https://www.w3.org/TR/html5/forms.html#file-upload-state-(type=file)
2017-09-20 19:07:23 +02:00
Eugen Rochko 26c797bf0d When accessing uncached media attachment, redownload it (#4955)
* When accessing uncached media attachment, redownload it

* Prevent re-download of rejected media
2017-09-16 03:01:45 +02:00
Eugen Rochko e9e271878e Make PreviewCard records reuseable between statuses (#4642)
* Make PreviewCard records reuseable between statuses

**Warning!** Migration truncates preview_cards tablec

* Allow a wider thumbnail for link preview, display it in horizontal layout (#4648)

* Delete preview cards files before truncating

* Rename old table instead of truncating it

* Add mastodon:maintenance:remove_deprecated_preview_cards

* Ignore deprecated_preview_cards in schema definition

* Fix null behaviour
2017-09-01 16:20:16 +02:00
Akihiko Odaki (@fn_aki@pawoo.net) 4a041cde77 Explicitly require MIME::Types (#4083) 2017-07-05 23:58:03 +02:00
Eugen Rochko 46af0c66b6 Fix #3675 - Adjust quality settings of converted GIFs to reduce filesize (#3723) 2017-06-13 00:51:48 +02:00
Matt Jankowski 0eb8b00cc9 Specs for cleanup workers (#3235)
* Add spec files for feed and media cleanup workers

* Add coverage for feed and media cleanup schedulers

* Clean up feed and media cleanup workers
2017-05-22 19:36:21 +02:00
Yamagishi Kazutoshi bd1f7d0b9c Fetch remote image using http.rb (#3114) 2017-05-18 15:43:10 +02:00
alpaca-tc 41295c0bbd Order by symbol value (#3077) 2017-05-16 03:35:17 +02:00
alpaca-tc 3ceb700ea2 Fixes unknown mime type (#2822) 2017-05-05 21:32:14 +02:00
yhirano e5eb070430 annotate models (#2697)
* add annotate to Gemfile

* rails g annotate:install

* configure annotate_models

* add schema info to models

* fix rubocop to add frozen_string_literal
2017-05-02 02:14:47 +02:00
Patrick Figel ef6f02c173 Set correct attachment type for rejected media (#2599)
In #2110, a new attachment type "unknown" was introduced for
attachments that were rejected due to a domain being blocked using
reject_media. However, the "type" field was never set to "unknown"
because a default value of "0" (image) is set for that column,
causing the `type.blank?` expression to always equal false.

This version uses type_changed? instead, causing the type to be set
to "unknown" unless a type has been explicitly set. This introduces
a small change in behaviour causing the type to be set to unknown
before paperclip calls `before_post_process`. Presumably this
behaviour is more appropriate than the current one because the
attachment type has not been determined by that point.

Included are new tests for `ProcessFeedService` and
`UpdateRemoteProfileService` which now check that remote media is
downloaded for non-blocked domains and is rejected for others.
2017-04-29 00:18:32 +02:00
Francis Chong 395608442d Add media dimensions (#2448)
* Fixes #1985

- add migration AddMediaAttachmentMeta, which add meta field to media_attachments
- before saving attachment, set file meta if needed
- add meta in api

* add spec

* align the “size” format for image and video

* fix code climate

* fixes media_attachment_spec.rb
2017-04-26 03:48:12 +02:00
Eugen 87f7a3922c Punycode URI normalization (#2370)
* Fix #2119 - Whenever about to send a HTTP request, normalize the URI

* Add test for IDN request in FetchLinkCardService

* Perform IDN normalization on domains before they are stored in the DB
2017-04-25 02:47:31 +02:00
Eugen 5849eac93e Fix #2108 - Fix gif uploads (#2171)
* Fix #2108 - Fix gif uploads
Add specs for media attachment gifv conversion

* Add ffmpeg to travis

* Make travis install ffmpeg, not libav

* Switch travis to trusty
2017-04-19 23:21:00 +02:00