Merge branch 'main' into glitch-soc/merge-upstream

CHANGELOG.md

@@ -6,7 +6,7 @@ All notable changes to this project will be documented in this file.
 ## Unreleased
 ### Added
 
-- **Add follow recommendations for onboarding** ([Gargron](https://github.com/tootsuite/mastodon/pull/15945), [Gargron](https://github.com/tootsuite/mastodon/pull/16161), [Gargron](https://github.com/tootsuite/mastodon/pull/16060), [Gargron](https://github.com/tootsuite/mastodon/pull/16077), [Gargron](https://github.com/tootsuite/mastodon/pull/16078), [Gargron](https://github.com/tootsuite/mastodon/pull/16160), [Gargron](https://github.com/tootsuite/mastodon/pull/16079), [noellabo](https://github.com/tootsuite/mastodon/pull/16044), [noellabo](https://github.com/tootsuite/mastodon/pull/16045), [Gargron](https://github.com/tootsuite/mastodon/pull/16152), [Gargron](https://github.com/tootsuite/mastodon/pull/16153), [ClearlyClaire](https://github.com/tootsuite/mastodon/pull/16082), [ClearlyClaire](https://github.com/tootsuite/mastodon/pull/16173), [ClearlyClaire](https://github.com/tootsuite/mastodon/pull/16159))
+- **Add follow recommendations for onboarding** ([Gargron](https://github.com/tootsuite/mastodon/pull/15945), [Gargron](https://github.com/tootsuite/mastodon/pull/16161), [Gargron](https://github.com/tootsuite/mastodon/pull/16060), [Gargron](https://github.com/tootsuite/mastodon/pull/16077), [Gargron](https://github.com/tootsuite/mastodon/pull/16078), [Gargron](https://github.com/tootsuite/mastodon/pull/16160), [Gargron](https://github.com/tootsuite/mastodon/pull/16079), [noellabo](https://github.com/tootsuite/mastodon/pull/16044), [noellabo](https://github.com/tootsuite/mastodon/pull/16045), [Gargron](https://github.com/tootsuite/mastodon/pull/16152), [Gargron](https://github.com/tootsuite/mastodon/pull/16153), [ClearlyClaire](https://github.com/tootsuite/mastodon/pull/16082), [ClearlyClaire](https://github.com/tootsuite/mastodon/pull/16173), [ClearlyClaire](https://github.com/tootsuite/mastodon/pull/16159), [ClearlyClaire](https://github.com/tootsuite/mastodon/pull/16189))
   - Tutorial on first web UI launch has been replaced with follow suggestions
   - Follow suggestions take user locale into account and are a mix of accounts most followed by currently active local users, and accounts that wrote the most shared/favourited posts in the last 30 days
   - Only accounts that have opted-in to being discoverable from their profile settings, and that do not require follow requests, will be suggested
@@ -23,7 +23,7 @@ All notable changes to this project will be documented in this file.
   - The dashboard will now warn you if you some Sidekiq queues are not being processed, if you have not defined any server rules, or if you forgot to run database migrations from the latest Mastodon upgrade
 - Add inline description of moderation actions in admin UI ([ClearlyClaire](https://github.com/tootsuite/mastodon/pull/15792))
 - Add "recommended" label to activity/peers API toggles in admin UI ([Gargron](https://github.com/tootsuite/mastodon/pull/16081))
-- Add joined date to profiles in web UI ([Gargron](https://github.com/tootsuite/mastodon/pull/16169))
+- Add joined date to profiles in web UI ([Gargron](https://github.com/tootsuite/mastodon/pull/16169), [rinsuki](https://github.com/tootsuite/mastodon/pull/16186))
 - Add transition to media modal background in web UI ([mkljczk](https://github.com/tootsuite/mastodon/pull/15843))
 - Add option to opt-out of unread notification markers in web UI ([ClearlyClaire](https://github.com/tootsuite/mastodon/pull/15842))
 - Add borders to 📱, 🚲, and 📲 emojis in web UI ([ClearlyClaire](https://github.com/tootsuite/mastodon/pull/15794), [ClearlyClaire](https://github.com/tootsuite/mastodon/pull/16035))
@@ -44,6 +44,7 @@ All notable changes to this project will be documented in this file.
   - This param allows an app to control from whom notifications should be delivered as push notifications to the app
 - Add `details` to error response for `POST /api/v1/accounts` in REST API ([Gargron](https://github.com/tootsuite/mastodon/pull/15803))
   - This attribute allows an app to display more helpful information to the user about why the sign-up did not succeed
+- Add `SIDEKIQ_REDIS_URL` and related environment variables to optionally use a separate Redis server for Sidekiq ([noellabo](https://github.com/tootsuite/mastodon/pull/16188))
 
 ### Changed
 
@@ -120,6 +121,7 @@ All notable changes to this project will be documented in this file.
 - Fix trying to fetch key from empty URI when verifying HTTP signature ([Gargron](https://github.com/tootsuite/mastodon/pull/16100))
 - Fix `tootctl maintenance fix-duplicates` failures ([ClearlyClaire](https://github.com/tootsuite/mastodon/pull/15923), [ClearlyClaire](https://github.com/tootsuite/mastodon/pull/15515))
 - Fix error when removing status caused by race condition ([Gargron](https://github.com/tootsuite/mastodon/pull/16099))
+- Fix blocking someone not clearing up list feeds ([ClearlyClaire](https://github.com/tootsuite/mastodon/pull/16205))
 - Fix misspelled URLs character counting ([ClearlyClaire](https://github.com/tootsuite/mastodon/pull/15382))
 - Fix Sidekiq hanging forever due to a Resolv bug in Ruby 2.7.3 ([ClearlyClaire](https://github.com/tootsuite/mastodon/pull/16157))
 - Fix edge case where follow limit interferes with accepting a follow ([ClearlyClaire](https://github.com/tootsuite/mastodon/pull/16098))

app/javascript/mastodon/actions/boosts.js

@@ -11,7 +11,7 @@ export function initBoostModal(props) {
 
     dispatch({
       type: BOOSTS_INIT_MODAL,
-      privacy
+      privacy,
     });
 
     dispatch(openModal('BOOST', props));

app/javascript/mastodon/actions/importer/normalizer.js

@@ -60,6 +60,7 @@ export function normalizeStatus(status, normalOldStatus) {
     normalStatus.search_index = normalOldStatus.get('search_index');
     normalStatus.contentHtml = normalOldStatus.get('contentHtml');
     normalStatus.spoilerHtml = normalOldStatus.get('spoilerHtml');
+    normalStatus.spoiler_text = normalOldStatus.get('spoiler_text');
     normalStatus.hidden = normalOldStatus.get('hidden');
   } else {
     // If the status has a CW but no contents, treat the CW as if it were the

app/javascript/mastodon/features/follow_recommendations/index.js

@@ -76,7 +76,7 @@ class FollowRecommendations extends ImmutablePureComponent {
 
     return (
       <Column>
-        <div className='scrollable'>
+        <div className='scrollable follow-recommendations-container'>
           <div className='column-title'>
             <Logo />
             <h3><FormattedMessage id='follow_recommendations.heading' defaultMessage="Follow people you'd like to see posts from! Here are some suggestions." /></h3>

app/javascript/mastodon/features/notifications/containers/notification_container.js

@@ -2,7 +2,6 @@ import { connect } from 'react-redux';
 import { makeGetNotification, makeGetStatus } from '../../../selectors';
 import Notification from '../components/notification';
 import { initBoostModal } from '../../../actions/boosts';
-import { openModal } from '../../../actions/modal';
 import { mentionCompose } from '../../../actions/compose';
 import {
   reblog,

app/javascript/mastodon/features/ui/components/focal_point_modal.js

@@ -309,7 +309,7 @@ class FocalPointModal extends ImmutablePureComponent {
     return (
       <div className='modal-root__modal report-modal' style={{ maxWidth: 960 }}>
         <div className='report-modal__target'>
-          <IconButton className='media-modal__close' title={intl.formatMessage(messages.close)} icon='times' onClick={onClose} size={16} />
+          <IconButton className='report-modal__close' title={intl.formatMessage(messages.close)} icon='times' onClick={onClose} size={20} />
           <FormattedMessage id='upload_modal.edit_media' defaultMessage='Edit media' />
         </div>
 

app/javascript/mastodon/features/ui/components/report_modal.js

@@ -91,7 +91,7 @@ class ReportModal extends ImmutablePureComponent {
     return (
       <div className='modal-root__modal report-modal'>
         <div className='report-modal__target'>
-          <IconButton className='media-modal__close' title={intl.formatMessage(messages.close)} icon='times' onClick={onClose} size={16} />
+          <IconButton className='report-modal__close' title={intl.formatMessage(messages.close)} icon='times' onClick={onClose} size={20} />
           <FormattedMessage id='report.target' defaultMessage='Report {target}' values={{ target: <strong>{account.get('acct')}</strong> }} />
         </div>
 

app/javascript/styles/mastodon/components.scss

@@ -2508,13 +2508,20 @@ a.account__display-name {
   }
 }
 
+.follow-recommendations-container {
+  display: flex;
+  flex-direction: column;
+}
+
 .column-actions {
   display: flex;
-  align-items: center;
+  align-items: start;
   justify-content: center;
   padding: 40px;
   padding-top: 40px;
   padding-bottom: 200px;
+  flex-grow: 1;
+  position: relative;
 
   &__background {
     position: absolute;
@@ -5297,9 +5304,10 @@ a.status-card.compact:hover {
 .report-modal__target {
   padding: 15px;
 
-  .media-modal__close {
+  .report-modal__close {
-    top: 14px;
+    position: absolute;
-    right: 15px;
+    top: 10px;
+    right: 10px;
   }
 }
 

app/lib/activitypub/activity/create.rb

@@ -46,7 +46,7 @@ class ActivityPub::Activity::Create < ActivityPub::Activity
     return reject_payload! if unsupported_object_type? || invalid_origin?(object_uri) || tombstone_exists? || !related_to_local_activity?
 
     lock_or_fail("create:#{object_uri}") do
-      return if delete_arrived_first?(object_uri) || poll_vote? # rubocop:disable Lint/NonLocalExitFromIterator
+      return if delete_arrived_first?(object_uri) || poll_vote?
 
       @status = find_existing_status
 

app/models/concerns/account_avatar.rb

@@ -21,7 +21,7 @@ module AccountAvatar
     has_attached_file :avatar, styles: ->(f) { avatar_styles(f) }, convert_options: { all: '-strip' }, processors: [:lazy_thumbnail]
     validates_attachment_content_type :avatar, content_type: IMAGE_MIME_TYPES
     validates_attachment_size :avatar, less_than: LIMIT
-    remotable_attachment :avatar, LIMIT
+    remotable_attachment :avatar, LIMIT, suppress_errors: false
   end
 
   def avatar_original_url

app/models/concerns/account_header.rb

@@ -22,7 +22,7 @@ module AccountHeader
     has_attached_file :header, styles: ->(f) { header_styles(f) }, convert_options: { all: '-strip' }, processors: [:lazy_thumbnail]
     validates_attachment_content_type :header, content_type: IMAGE_MIME_TYPES
     validates_attachment_size :header, less_than: LIMIT
-    remotable_attachment :header, LIMIT
+    remotable_attachment :header, LIMIT, suppress_errors: false
   end
 
   def header_original_url

app/models/concerns/remotable.rb

@@ -28,9 +28,11 @@ module Remotable
           end
         rescue Mastodon::UnexpectedResponseError, HTTP::TimeoutError, HTTP::ConnectionError, OpenSSL::SSL::SSLError => e
           Rails.logger.debug "Error fetching remote #{attachment_name}: #{e}"
+          public_send("#{attachment_name}=", nil) if public_send("#{attachment_name}_file_name").present?
           raise e unless suppress_errors
         rescue Paperclip::Errors::NotIdentifiedByImageMagickError, Addressable::URI::InvalidURIError, Mastodon::HostValidationError, Mastodon::LengthValidationError, Paperclip::Error, Mastodon::DimensionsValidationError, Mastodon::StreamValidationError => e
           Rails.logger.debug "Error fetching remote #{attachment_name}: #{e}"
+          public_send("#{attachment_name}=", nil) if public_send("#{attachment_name}_file_name").present?
         end
 
         nil

app/serializers/activitypub/actor_serializer.rb

@@ -177,7 +177,7 @@ class ActivityPub::ActorSerializer < ActivityPub::Serializer
     end
 
     def href
-      explore_hashtag_url(object)
+      tag_url(object)
     end
 
     def name

app/services/activitypub/process_account_service.rb

@@ -106,8 +106,16 @@ class ActivityPub::ProcessAccountService < BaseService
   end
 
   def set_fetchable_attributes!
+    begin
       @account.avatar_remote_url = image_url('icon') || '' unless skip_download?
+    rescue Mastodon::UnexpectedResponseError, HTTP::TimeoutError, HTTP::ConnectionError, OpenSSL::SSL::SSLError
+      RedownloadAvatarWorker.perform_in(rand(30..600).seconds, @account.id)
+    end
+    begin
       @account.header_remote_url = image_url('image') || '' unless skip_download?
+    rescue Mastodon::UnexpectedResponseError, HTTP::TimeoutError, HTTP::ConnectionError, OpenSSL::SSL::SSLError
+      RedownloadHeaderWorker.perform_in(rand(30..600).seconds, @account.id)
+    end
     @account.statuses_count    = outbox_total_items    if outbox_total_items.present?
     @account.following_count   = following_total_items if following_total_items.present?
     @account.followers_count   = followers_total_items if followers_total_items.present?

app/workers/redownload_avatar_worker.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+class RedownloadAvatarWorker
+  include Sidekiq::Worker
+  include ExponentialBackoff
+  include JsonLdHelper
+
+  sidekiq_options queue: 'pull', retry: 7
+
+  def perform(id)
+    account = Account.find(id)
+
+    return if account.suspended? || DomainBlock.rule_for(account.domain)&.reject_media?
+    return if account.avatar_remote_url.blank? || account.avatar_file_name.present?
+
+    account.reset_avatar!
+    account.save!
+  rescue ActiveRecord::RecordNotFound
+    # Do nothing
+  rescue Mastodon::UnexpectedResponseError => e
+    response = e.response
+
+    if response_error_unsalvageable?(response)
+      # Give up
+    else
+      raise e
+    end
+  end
+end

app/workers/redownload_header_worker.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+class RedownloadHeaderWorker
+  include Sidekiq::Worker
+  include ExponentialBackoff
+  include JsonLdHelper
+
+  sidekiq_options queue: 'pull', retry: 7
+
+  def perform(id)
+    account = Account.find(id)
+
+    return if account.suspended? || DomainBlock.rule_for(account.domain)&.reject_media?
+    return if account.header_remote_url.blank? || account.header_file_name.present?
+
+    account.reset_header!
+    account.save!
+  rescue ActiveRecord::RecordNotFound
+    # Do nothing
+  rescue Mastodon::UnexpectedResponseError => e
+    response = e.response
+
+    if response_error_unsalvageable?(response)
+      # Give up
+    else
+      raise e
+    end
+  end
+end

config/brakeman.ignore

@@ -7,7 +7,7 @@
       "check_name": "SQL",
       "message": "Possible SQL injection",
       "file": "app/models/report.rb",
-      "line": 112,
+      "line": 113,
       "link": "https://brakemanscanner.org/docs/warning_types/sql_injection/",
       "code": "Admin::ActionLog.from(\"(#{[Admin::ActionLog.where(:target_type => \"Report\", :target_id => id, :created_at => ((created_at..updated_at))).unscope(:order), Admin::ActionLog.where(:target_type => \"Account\", :target_id => target_account_id, :created_at => ((created_at..updated_at))).unscope(:order), Admin::ActionLog.where(:target_type => \"Status\", :target_id => status_ids, :created_at => ((created_at..updated_at))).unscope(:order)].map do\n \"(#{query.to_sql})\"\n end.join(\" UNION ALL \")}) AS admin_action_logs\")",
       "render_path": null,
@@ -67,7 +67,7 @@
       "check_name": "SQL",
       "message": "Possible SQL injection",
       "file": "app/models/account.rb",
-      "line": 491,
+      "line": 479,
       "link": "https://brakemanscanner.org/docs/warning_types/sql_injection/",
       "code": "find_by_sql([\"          WITH first_degree AS (\\n            SELECT target_account_id\\n            FROM follows\\n            WHERE account_id = ?\\n            UNION ALL\\n            SELECT ?\\n          )\\n          SELECT\\n            accounts.*,\\n            (count(f.id) + 1) * ts_rank_cd(#{textsearch}, #{query}, 32) AS rank\\n          FROM accounts\\n          LEFT OUTER JOIN follows AS f ON (accounts.id = f.account_id AND f.target_account_id = ?)\\n          WHERE accounts.id IN (SELECT * FROM first_degree)\\n            AND #{query} @@ #{textsearch}\\n            AND accounts.suspended_at IS NULL\\n            AND accounts.moved_to_account_id IS NULL\\n          GROUP BY accounts.id\\n          ORDER BY rank DESC\\n          LIMIT ? OFFSET ?\\n\".squish, account.id, account.id, account.id, limit, offset])",
       "render_path": null,
@@ -120,6 +120,26 @@
       "confidence": "High",
       "note": ""
     },
+    {
+      "warning_type": "Mass Assignment",
+      "warning_code": 105,
+      "fingerprint": "874be88fedf4c680926845e9a588d3197765a6ccbfdd76466b44cc00151c612e",
+      "check_name": "PermitAttributes",
+      "message": "Potentially dangerous key allowed for mass assignment",
+      "file": "app/controllers/api/v1/admin/reports_controller.rb",
+      "line": 78,
+      "link": "https://brakemanscanner.org/docs/warning_types/mass_assignment/",
+      "code": "params.permit(:resolved, :account_id, :target_account_id)",
+      "render_path": null,
+      "location": {
+        "type": "method",
+        "class": "Api::V1::Admin::ReportsController",
+        "method": "filter_params"
+      },
+      "user_input": ":account_id",
+      "confidence": "High",
+      "note": ""
+    },
     {
       "warning_type": "SQL Injection",
       "warning_code": 0,
@@ -127,7 +147,7 @@
       "check_name": "SQL",
       "message": "Possible SQL injection",
       "file": "app/models/account.rb",
-      "line": 460,
+      "line": 448,
       "link": "https://brakemanscanner.org/docs/warning_types/sql_injection/",
       "code": "find_by_sql([\"        SELECT\\n          accounts.*,\\n          ts_rank_cd(#{textsearch}, #{query}, 32) AS rank\\n        FROM accounts\\n        WHERE #{query} @@ #{textsearch}\\n          AND accounts.suspended_at IS NULL\\n          AND accounts.moved_to_account_id IS NULL\\n        ORDER BY rank DESC\\n        LIMIT ? OFFSET ?\\n\".squish, limit, offset])",
       "render_path": null,
@@ -207,7 +227,7 @@
       "check_name": "SQL",
       "message": "Possible SQL injection",
       "file": "app/models/account.rb",
-      "line": 507,
+      "line": 495,
       "link": "https://brakemanscanner.org/docs/warning_types/sql_injection/",
       "code": "find_by_sql([\"          SELECT\\n            accounts.*,\\n            (count(f.id) + 1) * ts_rank_cd(#{textsearch}, #{query}, 32) AS rank\\n          FROM accounts\\n          LEFT OUTER JOIN follows AS f ON (accounts.id = f.account_id AND f.target_account_id = ?) OR (accounts.id = f.target_account_id AND f.account_id = ?)\\n          WHERE #{query} @@ #{textsearch}\\n            AND accounts.suspended_at IS NULL\\n            AND accounts.moved_to_account_id IS NULL\\n          GROUP BY accounts.id\\n          ORDER BY rank DESC\\n          LIMIT ? OFFSET ?\\n\".squish, account.id, account.id, limit, offset])",
       "render_path": null,
@@ -241,6 +261,6 @@
       "note": ""
     }
   ],
-  "updated": "2020-12-07 01:17:13 +0100",
+  "updated": "2021-05-11 20:22:27 +0900",
-  "brakeman_version": "4.10.0"
+  "brakeman_version": "5.0.1"
 }

config/locales/is.yml

@@ -272,7 +272,7 @@ is:
         create_domain_allow_html: "%{name} leyfði skýjasamband með léninu %{target}"
         create_domain_block_html: "%{name} útilokaði lénið %{target}"
         create_email_domain_block_html: "%{name} útilokaði póstlénið %{target}"
-        create_ip_block_html: "{name} útbjó reglu fyrir IP-vistfangið %{target}"
+        create_ip_block_html: "%{name} útbjó reglu fyrir IP-vistfangið %{target}"
         create_unavailable_domain_html: "%{name} stöðvaði afhendingu til lénsins %{target}"
         demote_user_html: "%{name} lækkaði notandann %{target} í tign"
         destroy_announcement_html: "%{name} eyddi tilkynninguni %{target}"
@@ -280,7 +280,7 @@ is:
         destroy_domain_allow_html: "%{name} bannaði skýjasamband með léninu %{target}"
         destroy_domain_block_html: "%{name} aflétti útilokun af léninu %{target}"
         destroy_email_domain_block_html: "%{name} aflétti útilokun af póstléninu %{target}"
-        destroy_ip_block_html: "{name} eyddi reglu fyrir IP-vistfangið %{target}"
+        destroy_ip_block_html: "%{name} eyddi reglu fyrir IP-vistfangið %{target}"
         destroy_status_html: "%{name} fjarlægði stöðufærslu frá %{target}"
         destroy_unavailable_domain_html: "%{name} hóf aftur afhendingu til lénsins %{target}"
         disable_2fa_user_html: "%{name} gerði kröfu um tveggja-þátta innskráningu óvirka fyrir notandann %{target}"
@@ -290,7 +290,7 @@ is:
         enable_user_html: "%{name} gerði innskráningu virka fyrir notandann %{target}"
         memorialize_account_html: "%{name} breytti notandaaðgangnum %{target} í minningargreinarsíðu"
         promote_user_html: "%{name} hækkaði notandann %{target} í tign"
-        remove_avatar_user_html: "{name} fjarlægði auðkennismynd af %{target}"
+        remove_avatar_user_html: "%{name} fjarlægði auðkennismynd af %{target}"
         reopen_report_html: "%{name} enduropnaði kæru %{target}"
         reset_password_user_html: "%{name} endurstillti lykilorð fyrir notandann %{target}"
         resolve_report_html: "%{name} leysti kæru %{target}"
@@ -300,7 +300,7 @@ is:
         unassigned_report_html: "%{name} fjarlægði úthlutun af kæru %{target}"
         unsensitive_account_html: "%{name} tók merkinguna viðkvæmt af myndefni frá %{target}"
         unsilence_account_html: "%{name} hætti að hylja notandaaðganginn %{target}"
-        unsuspend_account_html: "%{name} tók notandaaðganginn {target} úr bið"
+        unsuspend_account_html: "%{name} tók notandaaðganginn %{target} úr bið"
         update_announcement_html: "%{name} uppfærði tilkynningu %{target}"
         update_custom_emoji_html: "%{name} uppfærði tjáningartáknið %{target}"
         update_domain_block_html: "%{name} uppfærði lénalás fyrir %{target}"

lib/mastodon/version.rb

@@ -17,7 +17,7 @@ module Mastodon
     end
 
     def flags
-      'rc1'
+      'rc2'
     end
 
     def suffix

lib/paperclip/gif_transcoder.rb

@@ -108,9 +108,11 @@ module Paperclip
 
       final_file = Paperclip::Transcoder.make(file, options, attachment)
 
+      if options[:style] == :original
         attachment.instance.file_file_name    = File.basename(attachment.instance.file_file_name, '.*') + '.mp4'
         attachment.instance.file_content_type = 'video/mp4'
         attachment.instance.type              = MediaAttachment.types[:gifv]
+      end
 
       final_file
     end
@@ -118,7 +120,7 @@ module Paperclip
     private
 
     def needs_convert?
-      options[:style] == :original && GifReader.animated?(file.path)
+      GifReader.animated?(file.path)
     end
   end
 end