Add Podcast RSS feeds (#5487)

* Initial test implementation of Podcast RSS

This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.

I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.

* Update to pfeed-podcast 1.2.2

* Initial test implementation of Podcast RSS

This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.

I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.

* Update to pfeed-podcast 1.2.2

* Initial test implementation of Podcast RSS

This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.

I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.

* Update to pfeed-podcast 1.2.2

* Add correct feed image to RSS channel

* Prefer HLS videos for podcast RSS

Remove video/stream titles, add optional height attribute to podcast RSS

* Prefix podcast RSS images with root server URL

* Add optional video query support to include captions

* Add transcripts & person images to podcast RSS feed

* Prefer webseed/webtorrent files over HLS fragmented mp4s

* Experimentally adding podcast fields to basic config page

* Add validation for new basic config fields

* Don't include "content" in podcast feed, use full description for "description"

* Initial test implementation of Podcast RSS

This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.

I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.

* Update to pfeed-podcast 1.2.2

* Add correct feed image to RSS channel

* Prefer HLS videos for podcast RSS

Remove video/stream titles, add optional height attribute to podcast RSS

* Prefix podcast RSS images with root server URL

* Add optional video query support to include captions

* Add transcripts & person images to podcast RSS feed

* Prefer webseed/webtorrent files over HLS fragmented mp4s

* Experimentally adding podcast fields to basic config page

* Add validation for new basic config fields

* Don't include "content" in podcast feed, use full description for "description"

* Add medium/socialInteract to podcast RSS feeds. Use HTML for description

* Change base production image to bullseye, install prosody in image

* Add liveItem and trackers to Podcast RSS feeds

Remove height from alternateEnclosure, replaced with title.

* Clear Podcast RSS feed cache when live streams start/end

* Upgrade to Node 16

* Refactor clearCacheRoute to use ApiCache

* Remove unnecessary type hint

* Update dockerfile to node 16, install python-is-python2

* Use new file paths for captions/playlists

* Fix legacy videos in RSS after migration to object storage

* Improve method of identifying non-fragmented mp4s in podcast RSS feeds

* Don't include fragmented MP4s in podcast RSS feeds

* Add experimental support for podcast:categories on the podcast RSS item

* Fix undefined category when no videos exist

Allows for empty feeds to exist (important for feeds that might only go live)

* Add support for podcast:locked -- user has to opt in to show their email

* Use comma for podcast:categories delimiter

* Make cache clearing async

* Fix merge, temporarily test with pfeed-podcast

* Syntax changes

* Add EXT_MIMETYPE constants for captions

* Update & fix tests, fix enclosure mimetypes, remove admin email

* Add test for podacst:socialInteract

* Add filters hooks for podcast customTags

* Remove showdown, updated to pfeed-podcast 6.1.2

* Add 'action:api.live-video.state.updated' hook

* Avoid assigning undefined category to podcast feeds

* Remove nvmrc

* Remove comment

* Remove unused podcast config

* Remove more unused podcast config

* Fix MChannelAccountDefault type hint missed in merge

* Remove extra line

* Re-add newline in config

* Fix lint errors for isEmailPublic

* Fix thumbnails in podcast feeds

* Requested changes based on review

* Provide podcast rss 2.0 only on video channels

* Misc cleanup for a less messy PR

* Lint fixes

* Remove pfeed-podcast

* Add peertube version to new hooks

* Don't use query include, remove TODO

* Remove film medium hack

* Clear podcast rss cache before video/channel update hooks

* Clear podcast rss cache before video uploaded/deleted hooks

* Refactor podcast feed cache clearing

* Set correct person name from video channel

* Styling

* Fix tests

---------

Co-authored-by: Chocobozzz <me@florianbigard.com>

1 files changed, 0 insertions, 389 deletions

diff --git a/server/controllers/feeds.ts b/server/controllers/feeds.ts
deleted file mode 100644
index ef810a842..000000000
--- a/server/controllers/feeds.ts
+++ /dev/null
@@ -1,389 +0,0 @@
-import express from 'express'
-import { extname } from 'path'
-import { Feed } from '@peertube/feed'
-import { mdToOneLinePlainText, toSafeHtml } from '@server/helpers/markdown'
-import { getServerActor } from '@server/models/application/application'
-import { getCategoryLabel } from '@server/models/video/formatter/video-format-utils'
-import { MAccountDefault, MChannelBannerAccountDefault, MVideoFullLight } from '@server/types/models'
-import { ActorImageType, VideoInclude } from '@shared/models'
-import { buildNSFWFilter } from '../helpers/express-utils'
-import { CONFIG } from '../initializers/config'
-import { MIMETYPES, PREVIEWS_SIZE, ROUTE_CACHE_LIFETIME, WEBSERVER } from '../initializers/constants'
-import {
-  asyncMiddleware,
-  commonVideosFiltersValidator,
-  feedsFormatValidator,
-  setDefaultVideosSort,
-  setFeedFormatContentType,
-  videoCommentsFeedsValidator,
-  videoFeedsValidator,
-  videosSortValidator,
-  videoSubscriptionFeedsValidator
-} from '../middlewares'
-import { cacheRouteFactory } from '../middlewares/cache/cache'
-import { VideoModel } from '../models/video/video'
-import { VideoCommentModel } from '../models/video/video-comment'
-
-const feedsRouter = express.Router()
-
-const cacheRoute = cacheRouteFactory({
-  headerBlacklist: [ 'Content-Type' ]
-})
-
-feedsRouter.get('/feeds/video-comments.:format',
-  feedsFormatValidator,
-  setFeedFormatContentType,
-  cacheRoute(ROUTE_CACHE_LIFETIME.FEEDS),
-  asyncMiddleware(videoFeedsValidator),
-  asyncMiddleware(videoCommentsFeedsValidator),
-  asyncMiddleware(generateVideoCommentsFeed)
-)
-
-feedsRouter.get('/feeds/videos.:format',
-  videosSortValidator,
-  setDefaultVideosSort,
-  feedsFormatValidator,
-  setFeedFormatContentType,
-  cacheRoute(ROUTE_CACHE_LIFETIME.FEEDS),
-  commonVideosFiltersValidator,
-  asyncMiddleware(videoFeedsValidator),
-  asyncMiddleware(generateVideoFeed)
-)
-
-feedsRouter.get('/feeds/subscriptions.:format',
-  videosSortValidator,
-  setDefaultVideosSort,
-  feedsFormatValidator,
-  setFeedFormatContentType,
-  cacheRoute(ROUTE_CACHE_LIFETIME.FEEDS),
-  commonVideosFiltersValidator,
-  asyncMiddleware(videoSubscriptionFeedsValidator),
-  asyncMiddleware(generateVideoFeedForSubscriptions)
-)
-
-// ---------------------------------------------------------------------------
-
-export {
-  feedsRouter
-}
-
-// ---------------------------------------------------------------------------
-
-async function generateVideoCommentsFeed (req: express.Request, res: express.Response) {
-  const start = 0
-  const video = res.locals.videoAll
-  const account = res.locals.account
-  const videoChannel = res.locals.videoChannel
-
-  const comments = await VideoCommentModel.listForFeed({
-    start,
-    count: CONFIG.FEEDS.COMMENTS.COUNT,
-    videoId: video ? video.id : undefined,
-    accountId: account ? account.id : undefined,
-    videoChannelId: videoChannel ? videoChannel.id : undefined
-  })
-
-  const { name, description, imageUrl } = buildFeedMetadata({ video, account, videoChannel })
-
-  const feed = initFeed({
-    name,
-    description,
-    imageUrl,
-    resourceType: 'video-comments',
-    queryString: new URL(WEBSERVER.URL + req.originalUrl).search
-  })
-
-  // Adding video items to the feed, one at a time
-  for (const comment of comments) {
-    const localLink = WEBSERVER.URL + comment.getCommentStaticPath()
-
-    let title = comment.Video.name
-    const author: { name: string, link: string }[] = []
-
-    if (comment.Account) {
-      title += ` - ${comment.Account.getDisplayName()}`
-      author.push({
-        name: comment.Account.getDisplayName(),
-        link: comment.Account.Actor.url
-      })
-    }
-
-    feed.addItem({
-      title,
-      id: localLink,
-      link: localLink,
-      content: toSafeHtml(comment.text),
-      author,
-      date: comment.createdAt
-    })
-  }
-
-  // Now the feed generation is done, let's send it!
-  return sendFeed(feed, req, res)
-}
-
-async function generateVideoFeed (req: express.Request, res: express.Response) {
-  const start = 0
-  const account = res.locals.account
-  const videoChannel = res.locals.videoChannel
-  const nsfw = buildNSFWFilter(res, req.query.nsfw)
-
-  const { name, description, imageUrl } = buildFeedMetadata({ videoChannel, account })
-
-  const feed = initFeed({
-    name,
-    description,
-    imageUrl,
-    resourceType: 'videos',
-    queryString: new URL(WEBSERVER.URL + req.url).search
-  })
-
-  const options = {
-    accountId: account ? account.id : null,
-    videoChannelId: videoChannel ? videoChannel.id : null
-  }
-
-  const server = await getServerActor()
-  const { data } = await VideoModel.listForApi({
-    start,
-    count: CONFIG.FEEDS.VIDEOS.COUNT,
-    sort: req.query.sort,
-    displayOnlyForFollower: {
-      actorId: server.id,
-      orLocalVideos: true
-    },
-    nsfw,
-    isLocal: req.query.isLocal,
-    include: req.query.include | VideoInclude.FILES,
-    hasFiles: true,
-    countVideos: false,
-    ...options
-  })
-
-  addVideosToFeed(feed, data)
-
-  // Now the feed generation is done, let's send it!
-  return sendFeed(feed, req, res)
-}
-
-async function generateVideoFeedForSubscriptions (req: express.Request, res: express.Response) {
-  const start = 0
-  const account = res.locals.account
-  const nsfw = buildNSFWFilter(res, req.query.nsfw)
-
-  const { name, description, imageUrl } = buildFeedMetadata({ account })
-
-  const feed = initFeed({
-    name,
-    description,
-    imageUrl,
-    resourceType: 'videos',
-    queryString: new URL(WEBSERVER.URL + req.url).search
-  })
-
-  const { data } = await VideoModel.listForApi({
-    start,
-    count: CONFIG.FEEDS.VIDEOS.COUNT,
-    sort: req.query.sort,
-    nsfw,
-
-    isLocal: req.query.isLocal,
-
-    hasFiles: true,
-    include: req.query.include | VideoInclude.FILES,
-
-    countVideos: false,
-
-    displayOnlyForFollower: {
-      actorId: res.locals.user.Account.Actor.id,
-      orLocalVideos: false
-    },
-    user: res.locals.user
-  })
-
-  addVideosToFeed(feed, data)
-
-  // Now the feed generation is done, let's send it!
-  return sendFeed(feed, req, res)
-}
-
-function initFeed (parameters: {
-  name: string
-  description: string
-  imageUrl: string
-  resourceType?: 'videos' | 'video-comments'
-  queryString?: string
-}) {
-  const webserverUrl = WEBSERVER.URL
-  const { name, description, resourceType, queryString, imageUrl } = parameters
-
-  return new Feed({
-    title: name,
-    description: mdToOneLinePlainText(description),
-    // updated: TODO: somehowGetLatestUpdate, // optional, default = today
-    id: webserverUrl,
-    link: webserverUrl,
-    image: imageUrl,
-    favicon: webserverUrl + '/client/assets/images/favicon.png',
-    copyright: `All rights reserved, unless otherwise specified in the terms specified at ${webserverUrl}/about` +
-    ` and potential licenses granted by each content's rightholder.`,
-    generator: `Toraifōsu`, // ^.~
-    feedLinks: {
-      json: `${webserverUrl}/feeds/${resourceType}.json${queryString}`,
-      atom: `${webserverUrl}/feeds/${resourceType}.atom${queryString}`,
-      rss: `${webserverUrl}/feeds/${resourceType}.xml${queryString}`
-    },
-    author: {
-      name: 'Instance admin of ' + CONFIG.INSTANCE.NAME,
-      email: CONFIG.ADMIN.EMAIL,
-      link: `${webserverUrl}/about`
-    }
-  })
-}
-
-function addVideosToFeed (feed: Feed, videos: VideoModel[]) {
-  for (const video of videos) {
-    const formattedVideoFiles = video.getFormattedVideoFilesJSON(false)
-
-    const torrents = formattedVideoFiles.map(videoFile => ({
-      title: video.name,
-      url: videoFile.torrentUrl,
-      size_in_bytes: videoFile.size
-    }))
-
-    const videoFiles = formattedVideoFiles.map(videoFile => {
-      const result = {
-        type: MIMETYPES.VIDEO.EXT_MIMETYPE[extname(videoFile.fileUrl)],
-        medium: 'video',
-        height: videoFile.resolution.id,
-        fileSize: videoFile.size,
-        url: videoFile.fileUrl,
-        framerate: videoFile.fps,
-        duration: video.duration
-      }
-
-      if (video.language) Object.assign(result, { lang: video.language })
-
-      return result
-    })
-
-    const categories: { value: number, label: string }[] = []
-    if (video.category) {
-      categories.push({
-        value: video.category,
-        label: getCategoryLabel(video.category)
-      })
-    }
-
-    const localLink = WEBSERVER.URL + video.getWatchStaticPath()
-
-    feed.addItem({
-      title: video.name,
-      id: localLink,
-      link: localLink,
-      description: mdToOneLinePlainText(video.getTruncatedDescription()),
-      content: toSafeHtml(video.description),
-      author: [
-        {
-          name: video.VideoChannel.getDisplayName(),
-          link: video.VideoChannel.Actor.url
-        }
-      ],
-      date: video.publishedAt,
-      nsfw: video.nsfw,
-      torrents,
-
-      // Enclosure
-      video: videoFiles.length !== 0
-        ? {
-          url: videoFiles[0].url,
-          length: videoFiles[0].fileSize,
-          type: videoFiles[0].type
-        }
-        : undefined,
-
-      // Media RSS
-      videos: videoFiles,
-
-      embed: {
-        url: WEBSERVER.URL + video.getEmbedStaticPath(),
-        allowFullscreen: true
-      },
-      player: {
-        url: WEBSERVER.URL + video.getWatchStaticPath()
-      },
-      categories,
-      community: {
-        statistics: {
-          views: video.views
-        }
-      },
-      thumbnails: [
-        {
-          url: WEBSERVER.URL + video.getPreviewStaticPath(),
-          height: PREVIEWS_SIZE.height,
-          width: PREVIEWS_SIZE.width
-        }
-      ]
-    })
-  }
-}
-
-function sendFeed (feed: Feed, req: express.Request, res: express.Response) {
-  const format = req.params.format
-
-  if (format === 'atom' || format === 'atom1') {
-    return res.send(feed.atom1()).end()
-  }
-
-  if (format === 'json' || format === 'json1') {
-    return res.send(feed.json1()).end()
-  }
-
-  if (format === 'rss' || format === 'rss2') {
-    return res.send(feed.rss2()).end()
-  }
-
-  // We're in the ambiguous '.xml' case and we look at the format query parameter
-  if (req.query.format === 'atom' || req.query.format === 'atom1') {
-    return res.send(feed.atom1()).end()
-  }
-
-  return res.send(feed.rss2()).end()
-}
-
-function buildFeedMetadata (options: {
-  videoChannel?: MChannelBannerAccountDefault
-  account?: MAccountDefault
-  video?: MVideoFullLight
-}) {
-  const { video, videoChannel, account } = options
-
-  let imageUrl = WEBSERVER.URL + '/client/assets/images/icons/icon-96x96.png'
-  let name: string
-  let description: string
-
-  if (videoChannel) {
-    name = videoChannel.getDisplayName()
-    description = videoChannel.description
-
-    if (videoChannel.Actor.hasImage(ActorImageType.AVATAR)) {
-      imageUrl = WEBSERVER.URL + videoChannel.Actor.Avatars[0].getStaticPath()
-    }
-  } else if (account) {
-    name = account.getDisplayName()
-    description = account.description
-
-    if (account.Actor.hasImage(ActorImageType.AVATAR)) {
-      imageUrl = WEBSERVER.URL + account.Actor.Avatars[0].getStaticPath()
-    }
-  } else if (video) {
-    name = video.name
-    description = video.description
-  } else {
-    name = CONFIG.INSTANCE.NAME
-    description = CONFIG.INSTANCE.DESCRIPTION
-  }
-
-  return { name, description, imageUrl }
-}