Commit Graph

270 Commits (466df752ba5bb32f712e3644aaae5edf0974d015)

Author SHA1 Message Date
Thibaut Girka c043697caa Merge branch 'master' into glitch-soc/merge-upstream 2018-08-19 09:27:18 +02:00
Eugen Rochko c28ed5d760 Do not use WHERE NOT IN in CopyStatusStats migration (#8281)
Fix #8275

As the batch operation progresses, the statuses_stats table grows,
and the WHERE NOT IN subquery becomes more expensive
2018-08-19 02:01:49 +02:00
Thibaut Girka 767cc06fe2 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	.circleci/config.yml
	app/controllers/authorize_follows_controller.rb
	app/javascript/packs/public.js

Moved new stuff from packs/public.js to core/public.js.
Added appropriate use_pack in new controllers.
2018-08-18 18:04:49 +02:00
Eugen Rochko 9e5285bfb1 Make CopyStatusStats migration use batches of 1000 to avoid locks (#8256) 2018-08-18 03:44:50 +02:00
Thibaut Girka 334f478db1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/models/status.rb
	db/migrate/20180528141303_fix_accounts_unique_index.rb
	db/schema.rb

Resolved by taking upstream changes (no real conflicts, just glitch-soc
specific code too close to actual changes).
2018-08-17 17:43:54 +02:00
Thibaut Girka ad7e23cc32 Split custom filter migration script using the post-deployment migration system
There were some concerns with the custom filter migration script dropping a table,
thus making it unsafe to run in a zero-downtime setting. Upstream introduced
a way to run migrations after deployment, so revisit the old migration script to
make use of this.
2018-08-16 20:30:09 +02:00
ThibG d329c3ab5e Fix FixAccountsUniqueIndex migration (#8212) 2018-08-15 20:23:12 +02:00
Eugen Rochko 659556ab1f Upgrade Doorkeeper to 4.4.1 (#8197) 2018-08-14 20:24:47 +02:00
Eugen Rochko 15c77f178f Fix null constraint violation in copy status stats migration (#8198) 2018-08-14 20:24:36 +02:00
Eugen Rochko bf5acc3eae Move status counters to separate table, count replies (#8104)
* Move status counters to separate table, count replies

* Migration to remove old counter columns from statuses table

* Fix schema file
2018-08-14 19:19:32 +02:00
Eugen Rochko 5e1e9753c3 Add post-deployment migration system (#8182)
Adopted from GitLab CE. Generate new migration with:

    rails g post_deployment_migration name_of_migration_here

By default they are run together with db:migrate. To not run them,
the env variable SKIP_POST_DEPLOYMENT_MIGRATIONS must be set

Code by Yorick Peterse <yorickpeterse@gmail.com>, see also:

83c8241160
2018-08-13 18:17:20 +02:00
Eugen Rochko 9a1cf4a558 Expect relays to answer with accept/reject (#8179) 2018-08-13 18:17:20 +02:00
ThibG d58242390e Make some migration script more robust (fixes #8007) (#8170)
Include a dummy Account class in the migration script containing only the
attributes relevant to the migration in order to not rely as much on the
codebase being in sync with the database schema.
2018-08-13 18:17:20 +02:00
Eugen Rochko 98184a889e Add post-deployment migration system (#8182)
Adopted from GitLab CE. Generate new migration with:

    rails g post_deployment_migration name_of_migration_here

By default they are run together with db:migrate. To not run them,
the env variable SKIP_POST_DEPLOYMENT_MIGRATIONS must be set

Code by Yorick Peterse <yorickpeterse@gmail.com>, see also:

83c8241160
2018-08-13 13:40:01 +02:00
Eugen Rochko 3fe1f95dd0 Expect relays to answer with accept/reject (#8179) 2018-08-12 18:16:26 +02:00
ThibG c74125df6a Make some migration script more robust (fixes #8007) (#8170)
Include a dummy Account class in the migration script containing only the
attributes relevant to the migration in order to not rely as much on the
codebase being in sync with the database schema.
2018-08-11 18:00:41 +02:00
Thibaut Girka 5ed6f6db1d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/controllers/accounts_controller.rb
	app/javascript/mastodon/locales/pl.json
	app/views/about/more.html.haml

Conflicts in `accounts_controller.rb` resolved by taking upstream's
version + our `use_pack`.

Conflicts in `pl.json` resolved by taking upstream's changes.

Conflicts in `aboute/more.html.haml` resolved by taking upstream's changes.
2018-08-10 16:22:31 +02:00
Eugen Rochko 6e3cb11f3c Public profile endorsements (accounts picked by profile owner) (#8146) 2018-08-09 09:56:53 +02:00
Thibaut Girka 4ec907e413 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	Dockerfile
	app/javascript/packs/common.js
	config/webpack/loaders/sass.js
	config/webpack/shared.js
	db/schema.rb
	package.json
	yarn.lock

A lot of the conflicts come from updating webpack.

Even though upstream deleted app/javascript/packs/common.js, I kept
glitch-soc's version as it unifies JS/CSS packs behavior across flavours.

Ported glitch changes to webpack 4.x
2018-07-16 14:50:42 +02:00
Akihiko Odaki eff09fbdff Fix index detection a migration to revert index change (#8026) 2018-07-15 14:56:00 +02:00
Eugen Rochko 171c20ce72 Add federation relay support (#7998)
* Add federation relay support

* Add admin UI for managing relays

* Include actor on relay-related activities

* Fix i18n
2018-07-13 02:16:06 +02:00
Thibaut Girka 262f7d147d Migrate glitch-soc keyword mutes to Mastodon's
Completely remove glitch-soc's Keyword Mutes, migrate
existing database records to CustomFilters.

Handling of client-side filters is still not implemented
in the glitch-soc front-end.
2018-07-09 19:19:23 +02:00
Thibaut Girka cf8121376b Merge branch 'master' into glitch-soc/tentative-merge
Conflicts:
	README.md
	app/controllers/statuses_controller.rb
	app/lib/feed_manager.rb
	config/navigation.rb
	spec/lib/feed_manager_spec.rb

Conflicts were resolved by taking both versions for each change.
This means the two filter systems (glitch-soc's keyword mutes and tootsuite's
custom filters) are in place, which will be changed in a follow-up commit.
2018-07-09 07:13:59 +02:00
ThibG 4c70bb6d6d Add option to not consider word boundaries when processing keyword filtering (#7975)
* Add option to not consider word boundaries when filtering phrases

* Add a few tests for keyword/phrase filtering
2018-07-09 02:22:09 +02:00
Eugen Rochko d878e3e945 Keyword/phrase filtering (#7905)
* Add keyword filtering

    GET|POST       /api/v1/filters
    GET|PUT|DELETE /api/v1/filters/:id

- Irreversible filters can drop toots from home or notifications
- Other filters can hide toots through the client app
- Filters use a phrase valid in particular contexts, expiration

* Make sure expired filters don't get applied client-side

* Add missing API methods

* Remove "regex filter" from column settings

* Add tests

* Add test for FeedManager

* Add CustomFilter test

* Add UI for managing filters

* Add streaming API event to allow syncing filters

* Fix tests
2018-06-29 15:34:36 +02:00
Thibaut Girka 44e49d2b52 Merge branch 'master' into glitch-soc/merge-upstream 2018-06-24 14:32:22 +02:00
takayamaki cab4aca9b2 fix : Disable reverted index changes (#7871) 2018-06-21 23:07:40 +02:00
Thibaut Girka 8bd7b9b586 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/models/user.rb

Resolved by adding :default_language to user settings fields
2018-06-21 20:49:57 +02:00
Eugen Rochko 499b213e8c Remove unused indexes (#7829) 2018-06-17 18:49:35 +02:00
Eugen Rochko 499a9f5f11 Change language opt-out to language opt-in (#7823)
* Switch filtered_languages to chosen_languages

* Adjust interface

* Remove unused translations
2018-06-17 13:54:02 +02:00
Thibaut Girka 3d6c594903 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/javascript/mastodon/initial_state.js
	db/schema.rb

Upstream added a new field to initial_state.
Not too sure about what happened with db/schema.rb though…
2018-06-15 20:51:39 +02:00
Eugen Rochko 907694df6a Add autofollow option to invites (#7805)
* Add autofollow option to invites

* Trigger CodeClimate rebuild
2018-06-15 18:00:23 +02:00
David Yip 4692eed7bd Merge branch 'master' into 454-allow-keyword-mutes-to-skip-mentions 2018-06-12 18:13:30 -05:00
Jenkins 695674b2fa Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-06-12 13:17:21 +00:00
nightpool 896589e034 Migration to cleanup blocked users that are still following (#7773)
* Migration to cleanup blocked users that are still following

* use follow directly, commit schema
2018-06-10 00:38:17 +02:00
Eugen Rochko cbec0de81e Migrate old web push subscriptions to ensure deliveries (#7764) 2018-06-09 13:33:04 +02:00
David Yip 2bb5842541 Thread scopes through #matches?. #454.
Also add an apply_to_mentions attribute on Glitch::KeywordMute, which is
used to calculate scope.  Next up: additions to the test suite to
demonstrate how scoping works.
2018-06-03 23:00:50 -05:00
David Yip 827d1ea864 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/javascript/mastodon/locales/en.json
 	app/javascript/mastodon/locales/ja.json
 	app/javascript/mastodon/locales/pl.json

The above conflicts appear to be a text conflict introduced by
glitch-soc's additional level of columns (i.e. moving a bunch of columns
under the Misc option).  They were resolved via accept-ours.
2018-06-02 16:15:36 -05:00
Eugen Rochko 5d2d05961c Wrong exception class: ActiveRecord::RecordNotUnique, not PG::UniqueViolation (#7688)
* Wrong exception class: ActiveRecord::RecordNotUnique, not PG::UniqueViolation

It's completely not obvious but PG::UniqueViolation is just a string inside the exception message, not the actual class of the exception

* Favourite does not have target_account_id
2018-05-31 17:22:33 +02:00
Eugen Rochko 7c50de2219 Improve account index migration (#7684)
* Improve account index migration

- Display more progress in stdout
- Catch PG::UniqueViolation when re-attributing favourites
- Skip callbacks and validations when re-attributing other relationships

* Use in_batches to reduce table lock-up during account merge

* Use #say_with_time to benchmark each deduplication
2018-05-31 17:09:09 +02:00
Jenkins a76f40890e Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-05-31 01:17:25 +00:00
Eugen Rochko 40ed681914 Fix nil error in migration (#7680)
Under rare circumstances the user record could have already been deleted before...
2018-05-31 02:30:37 +02:00
Yamagishi Kazutoshi abda29af50 Fix broken migrate (regression from #7658) (#7674) 2018-05-30 09:39:52 +02:00
Eugen Rochko 2ba4e0bf1a Deduplicate accounts and make unique username/domain index case-insensitive (#7658)
Fix #6937
Fix #6837
Fix #6667
2018-05-30 02:51:26 +02:00
Thibaut Girka efd2441d1f Merge branch 'master' into glitch-soc/merge 2018-05-14 20:51:50 +02:00
Akihiko Odaki fee0b5f372 Revert index change on statuses for api/v1/accounts account_id statuses (#7484) 2018-05-14 19:56:17 +02:00
Akihiko Odaki 28629d7921 Improve index on statuses for api/v1/accounts account_id statuses (#7476)
Queries with the combination of account_id, id, and visibility can be
categorized in three types:
1. Querying for public and unlisted to enumerate statuses visible to
anyone.
2. Querying for public, unlisted, and private to enumerate statuses
visible to follower.
3. Querying for direct to enumerate own direct statuses.

1 and 2 is covered by the index with condition 'visibility IN (0, 1, 2)'.
It would bring better performance in case that there are many direct
statuses.

The index with condition 'visibility = 3' is just for 3. It would be much
faster to query direct statuses thanks to this query.

The total size of those two indexes are expected to be smaller than the
deleted one because they are partial and does not have to cover all the
table.
2018-05-14 12:50:45 +02:00
Thibaut Girka d0b753db6b Merge branch 'master' into glitch-soc/merge
Conflicts:
	app/controllers/invites_controller.rb
	app/serializers/initial_state_serializer.rb
	config/locales/ko.yml
2018-05-11 18:12:42 +02:00
Eugen Rochko e86a4fe36b Add REST API for Web Push Notifications subscriptions (#7445)
- POST /api/v1/push/subscription
- PUT /api/v1/push/subscription
- DELETE /api/v1/push/subscription
- New OAuth scope: "push" (required for the above methods)
2018-05-11 11:49:12 +02:00
Thibaut Girka af504e62ff Merge branch 'master' into glitch-soc/master
Conflicts:
	app/models/account.rb
	app/views/accounts/_header.html.haml
2018-05-10 00:03:28 +02:00