Commit Graph

984 Commits (b5124e6d0426fc946f32273da3de7299f381a00b)

Author SHA1 Message Date
Alexander dff05cd398 improve status title (#8596)
* improve shown status title, useful for atom/rss

* use single quotes to satisfy codeclimate

* fix tests, make message more pretty

* fix tests

* fix codestyle

* fix codestyle

* remove atom_serializer_spec

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-01-11 06:58:16 +09:00
ThibG f52ef9b007 Allow blocking TLDs, and fix TLD blocks not being editable (#12805)
Fixes #12795

It was already possible to create domain blocks for TLDs, but those
weren't enforced, nor editable. This commit changes it so that they
are enforced and editable.
2020-01-08 22:42:05 +01:00
Eugen Rochko 413cb7e861 Fix base64-encoded file uploads not being possible (#12748)
Fix #3804, Fix #5776
2020-01-04 01:54:07 +01:00
Eugen Rochko c72c27756f Fix IDN mentions not being processed, IDN domains not being rendered (#12715)
This changes the REST API to return unicode domains in the `acct`
attribute instead of punycode, and to render unicode instead of
punycode on public HTML pages as well.

Fix #7812, fix #12246
2019-12-30 19:20:43 +01:00
ThibG ec3758bf82 Fix custom emoji category creation silently erroring out on duplicate category (#12647)
Instead, just re-use the existing category if any.

Fixes #12608
2019-12-18 16:56:59 +01:00
ThibG f5ac6f7801 Fixes featured hashtag setting page erroring out instead of rejecting invalid tags (#12436)
* Revert "Fix ignoring whole status because of one invalid hashtag (#11621)"

This reverts commit 123ffd0a45.

* Fix statuses being rejected because of invalid hashtag names

* Add spec for invalid hashtag names in statuses

* Add test for featured tags controller
2019-12-17 13:31:56 +01:00
scd31 27f1d5ee7d Increase max backup size (#12602)
* Increased max backup size

* partially reverted schema.rb
2019-12-17 13:30:58 +01:00
Alice Gaudon a285b3c63c Admin setting to disable default follows (#12566) 2019-12-16 23:55:50 +01:00
Eugen Rochko 0769ef8277 Add `tootctl media remove-orphans` (#12568) 2019-12-08 15:37:12 +01:00
Takeshi Umeda 58dd9698c3 Add basic support for group actors (#12071)
* Show badge on group actor in WebUI

* Do not notify in case of  by following group actor

* If you mention group actor, also mention group actor followers

* Relax characters that can be used in username (same as Application)

* Revert "Relax characters that can be used in username (same as Application)"

This reverts commit 7e10a137b878d0db1b5252c52106faef5e09ca4b.

* Delete display_name method
2019-12-04 20:36:33 +01:00
Yamagishi Kazutoshi b01cb4701e Update ESLint and RuboCop in Code Climate (#12534) 2019-12-02 18:25:43 +01:00
Mathieu Brunot 128a7ab7eb Add an LDAP Mail attribute config (#12053)
Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>
2019-12-01 18:52:21 +01:00
ThibG 60d26cabf0 Add follow_request notification type (#12198)
* Add follow_request notification type

The notification type already existed in the backend but was never pushed
to the front-end. This also means translation strings were also available
for the backend, from the notification mailer.

Unlike other notification types, these are off by default, to match what
I remember of Gargron's view on the topic: that follow requests should not
clutter notifications and should instead be reviewed at the user's own
leisure in the dedicated column.

Since follow requests have their own column, I've deemed it unnecessary to
add a specific tab for them in the notification quick filter.

* Show follow request link in single-column if there are pending requests, even if account isn't locked

* Push follow requests from notifications to the follow_requests list

* Offer to accept or reject follow request from the notification

* Redesign follow request notification
2019-12-01 17:25:29 +01:00
ThibG acb1927158 Only normalize local polls (#12515)
Before this patch, if remote poll options have leading or trailing spaces,
the information stored locally won't match them, causing federated voting to
fail.
2019-12-01 17:24:33 +01:00
Mathieu Brunot 462788cb12 Convert LDAP username (#12461)
*  Convert LDAP username #12021

Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>

* 🐛 Fix conversion var use

Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>

* 🐛 Fix LDAP uid conversion test

Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>

* 👌 Remove comments with ref to PR

Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>

* 👌 Remove unnecessary paranthesis

Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>

* 🔧 Move space in conversion string

Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>
2019-12-01 07:21:28 +01:00
ThibG 4b0a6d79dd Add ability to filter reports by target account domain (#12154)
* Add ability to filter reports by target account domain

* Reword by_target_domain label
2019-11-30 19:53:58 +01:00
ThibG c6c0b30608 Simplify SQL query when performing account search amongst followings (#12302) 2019-11-21 17:35:04 +01:00
ThibG 517c1cd062 Add bookmarks (#7107)
* Add backend support for bookmarks

Bookmarks behave like favourites, except they aren't shared with other
users and do not have an associated counter.

* Add spec for bookmark endpoints

* Add front-end support for bookmarks

* Introduce OAuth scopes for bookmarks

* Add bookmarks to archive takeout

* Fix migration

* Coding style fixes

* Fix rebase issue

* Update bookmarked_statuses to latest UI changes

* Update bookmark actions to properly reflect status changes in state

* Add bookmarks item to single-column layout

* Make active bookmarks red
2019-11-13 23:02:10 +01:00
ThibG 6b5aebf0ca Add abilityto add oneself to lists (#12271)
* Add ability to add oneself to lists

* Change search results to include oneself when searching through followers

* Mark follow relation as optional in ListAccount
2019-11-04 13:02:01 +01:00
ThibG 479d8ff47a Fix remote media descriptions being cut off at 420 chars (#12262)
* Fix remote media descriptions being cut off at 420 chars

Fixes #12258

* Fix tests
2019-11-04 13:00:16 +01:00
Faye Duxovni 6bca6dbc2d Add setting for whether to crop images in unexpanded toots (#12126) 2019-10-24 22:51:41 +02:00
Hugo Gameiro 9980046428 remove audio metadata (#12171) 2019-10-24 22:47:58 +02:00
Eugen Rochko cfb06d0565 Fix admin setting to auto-approve hashtags not affecting query (#12130)
Follow-up to #12122
2019-10-10 02:22:04 +02:00
Eugen Rochko d99a6145de Remove auto-silence behaviour from spam check (#12117)
Fix #12113
2019-10-09 07:11:23 +02:00
Eugen Rochko 2e07a901c5 Fix attachment not being re-downloaded even if file is not stored (#12125)
Change the behaviour of remotable concern. Previously, it would skip
downloading an attachment if the stored remote URL is identical to
the new one. Now it would not be skipped if the attachment is not
actually currently stored by Paperclip.
2019-10-09 07:10:46 +02:00
Eugen Rochko 001ce9a1a9 Add admin setting to auto-approve hashtags (#12122)
Change inaccurate labels on other admin settings
2019-10-09 00:30:15 +02:00
Eugen Rochko c453888616 Fix performance of home feed regeneration (#12084)
Fetching statuses from all followed accounts at once takes too long
within Postgres. Fetching them one by one and merging in Ruby
could be a lot less resource-intensive

Because the query for dynamically fetching the home timeline is so
heavy, we can no longer offer it when the home timeline is missing
2019-10-06 22:11:17 +02:00
Hugo Gameiro b302921eb1 add loglevel fatal to video and audio styles (#12088) 2019-10-06 19:48:26 +02:00
Eugen Rochko 746fae0bd1 Fix performance of GIF re-encoding (#12057)
* Change animated GIF detection to not shell out to ImageMagick

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Change video encoding parameters to limit to 10800 video frames

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Limit GIF image size further

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Always strip metadata from video files

* Fix code style issues
2019-10-03 01:09:12 +02:00
Eugen Rochko 1db0719dbb Fix needlessly indexing unsearchable statuses into ElasticSearch (#12041) 2019-10-02 20:04:46 +02:00
Eugen Rochko ced431f02f Fix account counters being overwritten by parallel writes (#12045) 2019-10-02 04:59:37 +02:00
Eugen Rochko f77337daaf Fix records not being indexed sometimes (#12024)
It's possible that after commit callbacks were not firing when
exceptions occurred in the process. Also, the default Sidekiq
strategy does not push indexing jobs immediately, which is not
necessary and could be part of the issue too.
2019-10-01 01:19:11 +02:00
ThibG f088e090e4 Add voters count support (#11917)
* Add voters count to polls

* Add ActivityPub serialization and parsing of voters count

* Add support for voters count in WebUI

* Move incrementation of voters count out of redis lock

* Reword “voters” to “people”
2019-09-29 22:58:01 +02:00
Eugen Rochko f5c71a6cd2 Add (back) option to set redirect notice on account without moving followers (#11994)
Fix #11913
2019-09-29 05:03:19 +02:00
Eugen Rochko d8f09d5853 Fix unscoped being used in the wrong place (#11987)
Regression from #11982
2019-09-28 05:23:32 +02:00
Eugen Rochko 386dc65671 Fix preview card image not being re-fetched even if link is re-posted (#11981)
Fix #11956
2019-09-28 01:33:16 +02:00
Eugen Rochko 6baaceefcd Fix index not being used in Status.reblogs_map (#11982)
Regression from #11623
2019-09-28 01:33:02 +02:00
Eugen Rochko 66d3e13777 Add `exclude_unreviewed` param to `GET /api/v2/search` REST API (#11977)
Make it so normal search returns even unreviewed matches, but
autosuggestions do not.

Fix #11960
2019-09-28 01:02:21 +02:00
abcang 2c43d538dc Validate Web::PushSubscription (#11971) 2019-09-27 15:24:13 +02:00
Eugen Rochko 67796a267d Fix relays UI being available in whitelist/secure mode (#11963)
Fix relays UI referencing relay that is not functional
2019-09-27 02:13:34 +02:00
Eugen Rochko 3773115066 Fix authentication before 2FA challenge (#11943)
Regression from #11831
2019-09-24 04:35:36 +02:00
ThibG c596206f8c Show user what options they have voted (#11195)
* Add own_votes field to poll results in REST API

Fixes #10679

* Display user votes in WebUI

* Update styling

* Add vote checkmark to public pages
2019-09-22 14:15:18 +02:00
Takeshi Umeda 9267e4310d Accept acct starting with @ in account migration (#11907) 2019-09-21 09:11:38 +02:00
Takeshi Umeda 45fe523a18 Addition of update activity distribution by alias, minor correction (#11905)
* Addition of update activity distribution by alias, minor correction

* Distribute Update activity after adding alias
* Add uniqueness verification to alias uri
* accept acct starting with @

* fix double-quoted to single-quoted
2019-09-21 09:11:21 +02:00
Eugen Rochko a2d3728c36 Add account migration UI (#11846)
Fix #10736

- Change data export to be available for non-functional accounts
- Change non-functional accounts to include redirecting accounts
2019-09-19 20:58:19 +02:00
Eugen Rochko 61442032a2 Add table of contents to about page (#11885)
Move public domain blocks information to about page
2019-09-19 11:09:05 +02:00
Eugen Rochko 1781358bd9 Add password challenge to 2FA settings, e-mail notifications (#11878)
Fix #3961
2019-09-18 16:37:27 +02:00
Eugen Rochko 8904a4eb36 Fix TOTP codes not being filtered from logs during enabling/disabling (#11877)
Not a serious issue because they are meaningless past single use
2019-09-18 02:48:40 +02:00
han@highemelry 66241cfc7e Fix audit log error when custom emoji is copied from remote server (#11876) 2019-09-17 23:20:48 +02:00
mayaeh 2d79552c3c Fix the sample account icon of Profile directory to local only (#11872) 2019-09-17 10:06:43 +02:00
Eugen Rochko 68ce208da3 Fix SSO login not using existing account when e-mail is verified (#11862)
Fix #11472
2019-09-16 20:42:29 +02:00
mayaeh 566e85716f Add search and sort functions to hashtag admin UI (#11829)
* Add search and sort functions to hashtag admin UI

* Move scope processing from tags_controller to tag_filter

* Fix based on method naming conventions

* Fixed not to get 500 errors for invalid requests
2019-09-16 14:27:29 +02:00
Eugen Rochko 8eb0d880cb Fix 2FA challenge and password challenge for non-database users (#11831)
* Fix 2FA challenge not appearing for non-database users

Fix #11685

* Fix account deletion not working when using external login

Fix #11691
2019-09-15 21:08:39 +02:00
Eugen Rochko 45b759a2ca Fix hashtags being split by ZWNJ character (#11821)
Fix #11761
2019-09-13 16:01:26 +02:00
Eugen Rochko ec20fd2112 Change unlisted custom emoji to not appear in autosuggestions (#11818)
Fix #11669
2019-09-13 16:01:09 +02:00
Eugen Rochko 9301574423 Change max length of media descriptions from 420 to 1500 characters (#11819)
Fix #11658
2019-09-13 16:00:34 +02:00
Eugen Rochko 4291b74031 Change deletes to preserve soft-deleted statuses in unresolved reports (#11805)
Change all account actions except "none" to resolve all unresolved reports

Refactor `SuspendAccountService` to be more readable
2019-09-11 16:32:44 +02:00
Tao Bror Bojlén 0430cbec29 add admin setting for default search engine indexing (fix #11750) (#11804) 2019-09-11 08:44:58 +02:00
Eugen Rochko 8eab4349b4 Add retry for failed media downloads and `tootctl media refresh` (#11775) 2019-09-10 15:29:12 +02:00
Eugen Rochko 666c1a8a06 Change tootctl to use inline parallelization instead of Sidekiq (#11776)
- Remove --background option
- Add --concurrency(=5) option
- Add progress bars
2019-09-10 13:48:48 +02:00
Eugen Rochko 25fb124ee6 Add batch actions and categories to admin UI for custom emojis (#11793) 2019-09-09 22:44:17 +02:00
Eugen Rochko 7799c7c75f Add batch approve/reject for pending hashtags in admin UI (#11791) 2019-09-09 12:50:09 +02:00
Eugen Rochko 021a19deba Change half-life of trend decay (#11774) 2019-09-08 19:17:57 +02:00
Eugen Rochko 89989b6255 Add timeline read markers API (#11762)
Fix #4093
2019-09-06 13:55:51 +02:00
Eugen Rochko 436266ca27 Change trending hashtags to not disappear instantly after midnight (#11712) 2019-09-02 18:11:13 +02:00
umonaca 70e0c2f162 Add more media formats (#11687)
* Add more media formats

* Removed amr file type
2019-08-30 04:30:29 +02:00
Eugen Rochko c194c17cb7 Fix non-numbers being passed to Redis and causing an error (#11697) 2019-08-30 02:49:54 +02:00
Eugen Rochko ee9d0c0f38 Fix uncaught domain normalization error in remote follow (#11703) 2019-08-30 02:19:17 +02:00
Eugen Rochko 79922ae20f Add profile directory to web UI (#11688)
* Add profile directory to web UI

* Add a line of bio to the directory
2019-08-30 00:14:36 +02:00
Eugen Rochko 9959cdc79f Add option to include reported statuses in warning e-mail (#11639) 2019-08-23 22:37:23 +02:00
Eugen Rochko 32c781ec6b Add soft delete for statuses for instant deletes through API (#11623)
* Add soft delete for statuses to allow them to appear instant

* Allow reporting soft-deleted statuses and show them in the admin UI

* Change index for getting an account's statuses
2019-08-22 21:55:56 +02:00
Eugen Rochko e4605bbb4a Fix remote and staff-removed statuses leaving media behind for a day (#11638)
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
2019-08-22 04:17:12 +02:00
Eugen Rochko 123ffd0a45 Fix ignoring whole status because of one invalid hashtag (#11621)
Fix #11618
2019-08-19 20:36:44 +02:00
ThibG f17b0ff712 Add invite comments (#10465) 2019-08-19 11:40:42 +02:00
ThibG 867b9f3e9c Add public blocks to /about/blocks (#11298)
* Add automatic blocklist display in /about/blocks

Inspired by https://github.com/Gargron/mastodon.social-misc

* Add admin option to set who can see instance blocks

* Normalize locales files

* Rename “Sandbox” to “Silence” for consistency

* Disable /about/blocks when in whitelist mode

* Optionally display rationale for domain blocks

* Only display domain blocks that have user-facing limitations, and order them

* Redesign table of blocked domains to better handle long domain names and rationales

* Change domain blocks ordering now that rationales aren't displayed right away

* Only show explanation for block severities actually in use

* Reword instance block explanations and add disclaimer for public fetch mode
2019-08-19 11:35:48 +02:00
Eugen Rochko 3f5571301a Add `tootctl cache recount` command (#11597) 2019-08-18 14:55:03 +02:00
Eugen Rochko 848cb2a697 Add more accurate hashtag search (#11579)
* Add more accurate hashtag search

Using ElasticSearch to index hashtags with edge n-grams and score
them by usage within the last 7 days since last activity. Only
hashtags that have been reviewed and are listable can appear in
searches, unless they match the query exactly

* Fix search analyzer dropping non-ascii characters
2019-08-18 03:45:51 +02:00
Stanislas 8f7051ee33 Remove WebP support (#11589) 2019-08-17 22:04:15 +02:00
Eugen Rochko 6393a094c6 Add more accurate account search (#11537)
* Add more accurate account search

When ElasticSearch is available, a more accurate search is implemented:

- Using edge n-gram index for acct and display name
- Using asciifolding and cjk width normalization on display names
- Using Gaussian decay on account activity for additional scoring (recency)
- Using followers/friends ratio for additional scoring (spamminess)
- Using followers number for additional scoring (size)

The exact match precedence only takes effect when the input conforms
to the username format and the username part of it is complete, i.e.
when the user started typing the domain part.

* Support single-letter usernames

* Fix tests

* Fix not picking up account updates

* Add weights and normalization for scores, skip zero terms queries

* Use local counts for accounts index, adjust search parameters

* Fix mistakes

* Using updated_at of accounts is inadequate for remote accounts
2019-08-16 01:24:03 +02:00
Arnout Engelen bd43bd8925 Include max image dimensions in error (#11552) 2019-08-15 20:20:20 +02:00
kedama 7bb4d0e751 Fix some emojis in profile metadata labels are not emojified. (#11534) 2019-08-11 13:15:18 +02:00
Eugen Rochko 32ff86b6e5 Fix crash when saving invalid domain name (#11528)
Fix #7629
2019-08-08 23:04:19 +02:00
Eugen Rochko 85466a2b9d Fix weekly usage not being displayed correctly in hashtag admin UI (#11524)
Fix percentages in usage breakdown having too many digits

Change trending hashtags to only ask for review if a hashtag enters
the top 3 position in the set, since it's the only items shown in
the default web UI
2019-08-08 23:03:28 +02:00
Eugen Rochko 5948fbf969 Add GIF and WebP support for custom emojis (#11519)
Fix #11466
2019-08-08 23:03:09 +02:00
Eugen Rochko 10b486f627 Fix acct URIs with IDN domains not being resolved (#11520)
Fix #11494
2019-08-07 21:14:08 +02:00
ThibG 072e696aa1 Add domain block notes (#11515)
* Add database columns for adding notes to domain blocks/restrctions

* Add admin UI to set private and public comments when blocking a domain

* Add text for private and public comments on domain blocks

* Show domain block comments in admin UI

* Add comments to the domain block undo page

* Make UnblockDomainService more robust regarding upgraded domain blocks

* Allow editing domain blocks

* Rename button from “undo domain block” to “view domain block” in account admin UI

* Change test to unsilence silenced users from upgraded blocks
2019-08-07 20:20:23 +02:00
Eugen Rochko 25f549557b Fix trending tags returning less items than requested sometimes (#11513)
Add better sorting defaults to the hashtags admin UI

Add "not reviewed" filter to hashtags admin UI
2019-08-07 17:08:30 +02:00
Eugen Rochko fd352ec288 Fix account tags not being saved correctly (#11507)
* Fix account tags not being saved correctly

Regression from b838607

Fix Tag#discoverable not returning tags where listable is nil instead of true

Add notice when saving hashtags in admin UI

Change public hashtag and directory pages to return 404 for forbidden tags

* Remove unused locale string
2019-08-07 10:01:55 +02:00
Eugen Rochko b55fd4f12d Fix featured tags not finding the right tag on save (#11504)
Regression from b838607
2019-08-07 10:00:58 +02:00
Eugen Rochko 191392e5c3 Add trends UI with admin and user settings (#11502) 2019-08-06 17:57:52 +02:00
Eugen Rochko 05e9cd13eb Change admin UI for hashtags and add back whitelisted trends (#11490)
Fix #271

Add back the `GET /api/v1/trends` API with the caveat that it does
not return tags that have not been allowed to trend by the staff.

When a hashtag begins to trend (internally) and that hashtag has
not been previously reviewed by the staff, the staff is notified.

The new admin UI for hashtags allows filtering hashtags by where
they are used (e.g. in the profile directory), whether they have
been reviewed or are pending reviewal, they show by how many people
the hashtag is used in the directory, how many people used it
today, how many statuses with it have been created today, and it
allows fixing the name of the hashtag to make it more readable.

The disallowed hashtags feature has been reworked. It is now
controlled from the admin UI for hashtags instead of from
the file `config/settings.yml`
2019-08-05 19:54:29 +02:00
Eugen Rochko bb86c642a4 Fix tag score not being updated when the tag is trending (#11465) 2019-08-01 19:57:09 +02:00
Eugen Rochko b92e18080a Change hashtag search to only return results that have trended in the past (#11448)
* Change hashtag search to only return results that have trended in the past

A way to eliminate typos and other one-off "junk" results

* Fix excluding exact matches that don't have a score

* Fix tests
2019-07-30 20:29:50 +02:00
Eugen Rochko c90eba765d Add hashtag score for better sorting of autosuggestions (#11427)
* Add hashtag score for better sorting of autosuggestions

* Do not use `~<~` operator with no text_pattern_ops index
2019-07-30 13:10:40 +02:00
Eugen Rochko 825dc3ca22 Add whitelist mode (#11291) 2019-07-30 11:10:46 +02:00
Eugen Rochko c111bd01a4 Fix tag normalization and migration not removing duplicate tags (#11441)
Fix #11428
2019-07-29 20:40:21 +02:00
Eugen Rochko 172bf17555 Revert "Remove conversation URI (#11423)" (#11424)
This reverts commit 47bf2698c2.
2019-07-28 17:47:37 +02:00
Eugen Rochko 47bf2698c2 Remove conversation URI (#11423)
It is not part of ActivityPub and will free up a lot of space
2019-07-28 17:30:12 +02:00
Eugen Rochko b8386076c5 Change hashtags to preserve first-used casing (#11416) 2019-07-28 05:59:51 +02:00
Eugen Rochko 2854c8e4d6 Remove timestamps from converted images to make them deterministic (#11408) 2019-07-27 04:41:55 +02:00