Commit Graph

80 Commits (41923d1c6ba28df6652aefd9619acc33a5cf467a)

Author SHA1 Message Date
Eugen 6d6a429af8 Rewrite Atom generation from stream entries to use Ox instead of Nokogiri (#1124)
* Rewrite Atom generation from stream entries to use Ox instead of Nokogiri::Builder

StreamEntry is now limited to only statuses, which allows some optimization. Removed
extra queries on AccountsController#show. AtomSerializer instead of AtomBuilderHelper
used in AccountsController#show, StreamEntriesController#show, StreamEntryRenderer
and PubSubHubbub::DistributionWorker

PubSubHubbub::DistributionWorker moves n+1 DomainBlock query to PubSubHubbub::DeliveryWorker
instead.

All Salmon slaps that aren't based on StreamEntry still use AtomBuilderHelper and Nokogiri

* All Salmon slaps now use Ox instead of Nokogiri. No touch from status on account
2017-04-07 05:56:56 +02:00
Kurtis Rainbolt-Greene 03adb5d727 Mastodon isn't using jbuilder or sdoc, and it prevents an upgrade to 2.4.0 2017-04-04 21:31:02 -07:00
Samy KACIMI 5af0ecbcd9 alphebatically order test gem group as required by rubocop 2017-04-05 00:52:55 +02:00
Samy KACIMI 79ef756f64 fix rubocop issues 2017-04-05 00:47:17 +02:00
Samy KACIMI 81c76fe375 add more tests to models 2017-04-05 00:29:56 +02:00
Eugen Rochko 38b504b7a7 Remove sidekiq-merger 2017-04-04 14:28:57 +02:00
Eugen Rochko 82aaedec46 Reduce number of items in feeds, optimize regeneration worker slightly,
make regeneration worker unique, (only schedule/execute once at a time)
2017-04-04 13:58:34 +02:00
Eugen Rochko ce9df2fa82 Optimize filter methods in FeedManager a bit, use redis pipelining on merge/unmerge feed methods,
do not re-create a dynamic class on each feed push call, make sure redis-rb uses hiredis
2017-04-04 13:01:14 +02:00
Wonderfall 92d35c52d9 update Dockerfile 2017-04-03 00:46:44 +02:00
Eugen Rochko 5b12624847 Add proper error page for request timeouts 2017-04-02 19:43:44 +02:00
Eugen Rochko 7637386228 Upgrade Rails to 5.0.2 2017-03-22 15:37:03 +01:00
Eugen Rochko ad0d82d3ce Make account search blazing fast and rank followers/followees higher in the results 2017-03-17 20:48:14 +01:00
Eugen Rochko 3202bdd744 Fix #555 - Use a better URL parser 2017-02-22 19:35:11 +01:00
Eugen Rochko 9c88d1b99e Speed up capistrano deployments 2017-02-16 02:34:21 +01:00
Eugen Rochko 920ba5fc4e Fix #61 - Add list of blocked users to the UI; clean up failed push notifications API
Try to fix Travis CI setup
2017-02-05 19:18:11 +01:00
Eugen Rochko a2a85e8549 Add capistrano to development environment 2017-01-30 20:03:17 +01:00
Eugen Rochko 3f075c7794 API for apps to register for push notifications 2017-01-29 01:30:32 +01:00
Eugen Rochko ba192f12e3 Added optional two-factor authentication 2017-01-27 20:35:16 +01:00
Eugen Rochko cc5c1e5feb Removed accidental n+1 query from notifications, updated some deps,
improved how "show more" link looks
2017-01-26 14:52:07 +01:00
Eugen Rochko d4d56b8af0 Fix #365, 1/2 of #408 - replace rails_autolink with URI.regexp, run link_hashtags on simplified_format 2017-01-23 14:45:09 +01:00
Effy Elden 5fef55c11a Set remote dev environments to use letter_opener_web, add default .env.vagrant variables 2017-01-21 20:22:49 +11:00
Eugen Rochko f0de621e76 Fix #463 - Fetch and display previews of URLs using OpenGraph tags 2017-01-20 01:00:14 +01:00
Eugen Rochko f051c2e813 Fix statsd null backend not being initialized properly 2017-01-19 09:37:07 +01:00
Eugen Rochko 306eb6e9c9 Add optional StatsD performance tracking 2017-01-18 23:44:29 +01:00
Eugen Rochko b11fdc3ae3 Migrate from ledermann/rails-settings to rails-settings-cached which allows global settings
with YAML-defined defaults. Add admin page for editing global settings. Add "site_description"
setting that would show as a paragraph on the frontpage
2017-01-12 20:46:24 +01:00
Eugen Rochko 7951e7ffd5 Add ruby version to Gemfile, move devDependencies in package.json to dependencies,
fix bug in process feed service
2017-01-11 15:39:31 +01:00
Eugen Rochko c128fcee16 Update Rails to 5.0.1 2016-12-21 19:09:59 +01:00
Eugen Rochko 76ec907993 Improved admin UI 2016-12-13 13:42:10 +01:00
Eugen Rochko 5973ca3d11 Upgrade Paperclip to 5, AWS-SDK to 2, do not generate medium/small versions of avatars 2016-11-29 14:20:15 +01:00
Eugen Rochko cc70f28f19 Adding rack timeout of 30sec, PuSH jobs moved to push queue so they
can be processed separately
2016-11-29 02:07:14 +01:00
Eugen Rochko 8efa081f21 Remove Neo4J 2016-11-24 23:46:27 +01:00
Eugen Rochko 8e34bed7cc Mini Profiler not working well, remove it 2016-11-24 19:59:11 +01:00
Eugen Rochko fc90d38893 Moving some counter queries out of subqueries in the API 2016-11-22 22:59:54 +01:00
Eugen Rochko 4b5b953d42 Upgrade to latest Rails, which apparently improves ActionCable performance 2016-11-22 22:26:44 +01:00
Eugen Rochko 546c4718e7 Localizations for most server-side strings 2016-11-16 00:55:33 +01:00
Eugen Rochko fdc17bea58 Fix rubocop issues, introduce usage of frozen literal to improve performance 2016-11-15 16:56:29 +01:00
Eugen Rochko 09218d4c01 Use full-text search for autosuggestions 2016-11-12 14:36:10 +01:00
Eugen Rochko 448ab18a20 Replace setting custom CORS headers with rack-cors, set it on /oauth/token endpoint 2016-11-09 18:06:01 +01:00
Eugen Rochko d0d799f911 Replace react-skylight with own solution that centers images better 2016-11-07 18:23:36 +01:00
Eugen Rochko ea4170785a Downgrade aws gem as this one was incompatible with Paperclip 2016-11-06 20:55:57 +01:00
Eugen Rochko 3ab193bc3f Adding optional S3, fail-mastodon 2016-11-06 18:35:46 +01:00
Eugen Rochko 6471a548fe Add pghero for analyzing postgres performance in production 2016-11-04 18:35:04 +01:00
Eugen Rochko 53fb7b1aa8 Replace in-memory cache store with redis store, should decrease memory
footprint of each running process by 128mb
2016-11-02 13:30:52 +01:00
Eugen Rochko e21a3fe0cd Adding sync of follow relationships to Neo4J, accounts/suggestions API 2016-10-14 23:10:07 +02:00
Eugen Rochko 06016453bd Adding user settings (model and mailer), no form yet 2016-10-07 13:17:56 +02:00
Eugen Rochko 4909bbf415 Add logging for outgoing http requests 2016-10-02 14:58:06 +02:00
Eugen Rochko 57583f3fb8 Improve partial rendering and Haml performance (Haml->Hamlit) 2016-09-24 13:40:42 +02:00
Eugen Rochko 4bec613897 Fix #24 - Thread resolving for remote statuses
This is a big one, so let me enumerate:

Accounts as well as stream entry pages now contain Link headers that
reference the Atom feed and Webfinger URL for the former and Atom entry
for the latter. So you only need to HEAD those resources to get that
information, no need to download and parse HTML <link>s.

ProcessFeedService will now queue ThreadResolveWorker for each remote
status that it cannot find otherwise. Furthermore, entries are now
processed in reverse order (from bottom to top) in case a newer entry
references a chronologically previous one.

ThreadResolveWorker uses FetchRemoteStatusService to obtain a status
and attach the child status it was queued for to it.

FetchRemoteStatusService looks up the URL, first with a HEAD, tests
if it's an Atom feed, in which case it processes it directly. Next
for Link headers to the Atom feed, in which case that is fetched
and processed. Lastly if it's HTML, it is checked for <link>s to the Atom
feed, and if such is found, that is fetched and processed. The account for
the status is derived from author/name attribute in the XML and the hostname
in the URL (domain). FollowRemoteAccountService and ProcessFeedService
are used.

This means that potentially threads are resolved recursively until a dead-end
is encountered, however it is performed asynchronously over background jobs,
so it should be ok.
2016-09-21 01:50:31 +02:00
Eugen Rochko af375b5bac Update sidekiq, remove sinatra dependency 2016-09-18 13:40:30 +02:00
Eugen Rochko ce29624c6d Fixing image upload limits, allowing webm, merge/unmerge events trigger
timeline reload in UI, other small fixes
2016-09-12 18:22:43 +02:00