Commit Graph

659 Commits

Author SHA1 Message Date
Chocobozzz
3b5b050a98
Include transcription package in builds 2024-07-01 10:12:48 +02:00
Chocobozzz
1bfb791e05
Integrate transcription in PeerTube 2024-06-28 08:44:58 +02:00
lutangar
ef14cf4a5c
feat(transcription): groundwork
chore: fiddling around some more

chore: add ctranslate2 and timestamped

chore: add performance markers

chore: refactor test

chore: change worflow name

chore: ensure Python3

chore(duration): convert to chai/mocha syntahx

chore(transcription): add individual tests for others transcribers

chore(transcription): implement formats test of all implementations

Also compare result of other implementation to the reference implementation

chore(transcription): add more test case with other language and models size and local model

chore(test): wip ctranslate 2 adapat

chore(transcription): wip transcript file and benchmark

chore(test): clean a bit

chore(test): clean a bit

chore(test): refacto timestamed spec

chore(test): update workflow

chore(test): fix glob expansion with sh

chore(test): extract some hw info

chore(test): fix async tests

chore(benchmark): add model info

feat(transcription): allow use of a local mode in timestamped-whisper

feat(transcription): extract run and profiling info in own value object

feat(transcription): extract run concept in own class an run more bench

chore(transcription): somplify run object only a uuid is now needed and add more benchmark scenario

docs(transcription): creates own package readme

docs(transcription): add local model usage

docs(transcription): update README

fix(transcription): use fr video for better comparison

chore(transcription): make openai comparison passed

docs(timestamped): clea

chore(transcription): change transcribers transcribe method signature

Introduce whisper builtin model.

fix(transcription): activate language detection

Forbid transcript creation without a language.
Add `languageDetection` flag to an engine and some assertions.

Fix an issue in `whisper-ctranslate2` :
https://github.com/Softcatala/whisper-ctranslate2/pull/93

chore(transcription): use PeerTube time helpers instead of custom ones

Update existing time function to output an integer number of seconds and add a ms human-readable time formatter with hints of tests.

chore(transcription): use PeerTube UUID helpers

chore(transcription): enable CER evaluation

Thanks to this recent fix in Jiwer <3
https://github.com/jitsi/jiwer/issues/873

chore(jiwer): creates JiWer package

I'm not very happy with the TranscriptFileEvaluator constructor... suggestions ?

chore(JiWer): add usage in README

docs(jiwer): update JiWer readme

chore(transcription): use FunMOOC video in fixtures

chore(transcription): add proper english video fixture

chore(transcription): use os tmp directory where relevant

chore(transcription): fix jiwer cli test reference.txt

chore(transcription): move benchmark out of tests

chore(transcription): remove transcription workflow

docs(transcription): add benchmark info

fix(transcription): use ms precision in other transcribers

chore(transcription): simplify most of the tests

chore(transcription): remove slashes when building path with join

chore(transcription): make fromPath method async

chore(transcription): assert path to model is a directory for CTranslate2 transcriber

chore(transcription): ctranslate2 assertion

chore(transcription): ctranslate2 assertion

chore(transcription): add preinstall script for Python dependencies

chore(transcription): add download and unzip utils functions

chore(transcription): add download and unzip utils functions

chore(transcription): download & unzip models fixtures

chore(transcription): zip

chore(transcription): raise download file test timeout

chore(transcription): simplify download file test

chore(transcription): add transcriptions test to CI

chore(transcription): raise test preconditions timeout

chore(transcription): run preinstall scripts before running ci

chore(transcription): create dedicated tmp folder for transcriber tests

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): use short video for local model test

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): setup verbosity based on NODE_ENV value
2024-06-28 08:43:40 +02:00
Chocobozzz
b10482e0e0
Fix lint 2024-06-27 14:34:18 +02:00
Chocobozzz
9772280e99
Upgrade to angular 18 & vite 2024-06-26 08:33:54 +02:00
Chocobozzz
a722194809
Update server dependencies 2024-06-21 14:39:50 +02:00
Chocobozzz
00b1adab86
Fix player back button label 2024-06-10 16:27:01 +02:00
Chocobozzz
b89f37a751
Add inspect node option to benchmark script
So we can easily profile the backend
2024-04-05 10:18:45 +02:00
Chocobozzz
f84243fc53
Ensure to release with appropriate node modules 2024-04-04 16:45:05 +02:00
Chocobozzz
5cb3e6a0b8
Use sessionId instead of IP to identify viewer
Breaking: YAML config `ip_view_expiration` is renamed `view_expiration`
Breaking: Views are taken into account after 10 seconds instead of 30
seconds (can be changed in YAML config)

Purpose of this commit is to get closer to other video platforms where
some platforms count views on play (mux, vimeo) or others use a very low
delay (instagram, tiktok)

We also want to improve the viewer identification, where we no longer
use the IP but the `sessionId` generated by the web browser. Multiple
viewers behind a NAT can now be able to be identified as independent
viewers (this method is also used by vimeo or mux)
2024-04-04 16:27:40 +02:00
Chocobozzz
a246c44504
Support tr locale 2024-03-26 14:25:12 +01:00
Chocobozzz
be19d9be34
Move email templates in assets directory 2024-03-20 09:15:07 +01:00
Chocobozzz
c727a34cb6
Prevent aborting another live session 2024-03-18 16:09:22 +01:00
Chocobozzz
1a8b20ba30
Less verbose on expected error 2024-03-18 11:28:43 +01:00
Chocobozzz
1048d1054b
Fix dev:client command 2024-03-15 16:49:55 +01:00
Chocobozzz
4b857f50c8
Anonymize IP stored in Redis too 2024-02-29 10:09:21 +01:00
Chocobozzz
8349c20c08
Update translations 2024-02-21 15:43:15 +01:00
Chocobozzz
8573e5a80a Implement user import/export in server 2024-02-21 13:49:08 +01:00
Chocobozzz
c97b8fd2ea
Add caption button to player control bar 2024-02-19 16:14:29 +01:00
Chocobozzz
12ea8f0dd1
Fix nginx serving private hls files 2024-01-18 09:09:31 +01:00
Chocobozzz
a043de80d2
Fix build concurrency issue 2023-12-18 09:36:08 +01:00
Chocobozzz
83e1e4dfb2
Fix backward compat upgrade script 2023-12-08 11:16:43 +01:00
Julien Rabier
7a953a6b2f Fix upgrade.sh when Peertube is installed outside the standard path (fixes #6063) 2023-11-29 11:03:44 +01:00
Chocobozzz
9f9522e865
Fix release script 2023-11-29 08:33:39 +01:00
Chocobozzz
9880d2adf5
Improve changelog 2023-11-08 09:39:44 +01:00
Chocobozzz
04bd31bc18
Add more live latency info in stats for nerds 2023-10-30 11:08:09 +01:00
Chocobozzz
0f21769205
Don't log http request to benchmark views api 2023-10-26 12:30:34 +02:00
Chocobozzz
b7f1bc0c33
Fix misc endpoints tests 2023-10-20 17:30:46 +02:00
Chocobozzz
f90db24233
Fix SEO and refactor HTML pages generation
* Split methods in multiple classes
 * Add JSONLD tags in embed too
 * Index embeds but use a canonical URL tag (targeting the watch page)
 * Remote objects don't include a canonical URL tag anymore. Instead we
   forbid indexation
 * Canonical URLs now use the official short URL (/w/, /w/p, /a, /c
   etc.)
2023-10-20 16:00:36 +02:00
Chocobozzz
71ba1ead4f
Fix types test 2023-10-11 15:29:11 +02:00
Chocobozzz
5a3d0650c9
server/server -> server/core 2023-10-04 15:13:25 +02:00
Chocobozzz
273d57023b
Update translations 2023-08-18 14:12:32 +02:00
Chocobozzz
0632cdda04
Fix build 2023-08-17 09:24:45 +02:00
Chocobozzz
3a4992633e
Migrate server to ESM
Sorry for the very big commit that may lead to git log issues and merge
conflicts, but it's a major step forward:

 * Server can be faster at startup because imports() are async and we can
   easily lazy import big modules
 * Angular doesn't seem to support ES import (with .js extension), so we
   had to correctly organize peertube into a monorepo:
    * Use yarn workspace feature
    * Use typescript reference projects for dependencies
    * Shared projects have been moved into "packages", each one is now a
      node module (with a dedicated package.json/tsconfig.json)
    * server/tools have been moved into apps/ and is now a dedicated app
      bundled and published on NPM so users don't have to build peertube
      cli tools manually
    * server/tests have been moved into packages/ so we don't compile
      them every time we want to run the server
 * Use isolatedModule option:
   * Had to move from const enum to const
     (https://www.typescriptlang.org/docs/handbook/enums.html#objects-vs-enums)
   * Had to explictely specify "type" imports when used in decorators
 * Prefer tsx (that uses esbuild under the hood) instead of ts-node to
   load typescript files (tests with mocha or scripts):
     * To reduce test complexity as esbuild doesn't support decorator
       metadata, we only test server files that do not import server
       models
     * We still build tests files into js files for a faster CI
 * Remove unmaintained peertube CLI import script
 * Removed some barrels to speed up execution (less imports)
2023-08-11 15:02:33 +02:00
Chocobozzz
9dfbf73576
Increase rate limits for benchmark 2023-07-27 14:18:59 +02:00
Chocobozzz
053ed7f5e6
Update code contributors 2023-07-26 09:49:53 +02:00
Chocobozzz
97f57e928f
Add custom db port when upgrading peertube 2023-07-11 13:43:41 +02:00
Chocobozzz
d732ec7b46
Rename "videos" config to "web-videos" 2023-07-11 11:23:51 +02:00
Chocobozzz
784e2ad5c3
Prefer web videos in favour of webtorrent 2023-07-11 09:21:13 +02:00
Chocobozzz
c3030e944a
Fix build 2023-07-10 16:41:08 +02:00
Chocobozzz
a1bd2b77d9
Remove webtorrent support from client 2023-07-10 16:08:28 +02:00
Chocobozzz
9c5cc50133
Add script to generate storyboards 2023-06-29 10:16:57 +02:00
Wicklow
40346ead2b
Feature/password protected videos (#5836)
* Add server endpoints

* Refactoring test suites

* Update server and add openapi documentation

* fix compliation and tests

* upload/import password protected video on client

* add server error code

* Add video password to update resolver

* add custom message when sharing pw protected video

* improve confirm component

* Add new alert in component

* Add ability to watch protected video on client

* Cannot have password protected replay privacy

* Add migration

* Add tests

* update after review

* Update check params tests

* Add live videos test

* Add more filter test

* Update static file privacy test

* Update object storage tests

* Add test on feeds

* Add missing word

* Fix tests

* Fix tests on live videos

* add embed support on password protected videos

* fix style

* Correcting data leaks

* Unable to add password protected privacy on replay

* Updated code based on review comments

* fix validator and command

* Updated code based on review comments
2023-06-29 09:48:55 +02:00
Chocobozzz
6403a6bd01
peertube runner tests in parallel 2023-05-19 13:33:27 +02:00
Chocobozzz
b30ad9888f
Add config file merging in upgrade script 2023-05-16 08:59:05 +02:00
Chocobozzz
902b16af03
Breaking: remove node 14 support 2023-05-09 10:02:54 +02:00
Chocobozzz
3a0c2a77b1 Enable external plugins to test the PR 2023-05-09 08:57:34 +02:00
Chocobozzz
1772b383de Add peertube runner cli 2023-05-09 08:57:34 +02:00
Chocobozzz
e592df48c7 Update i18n server strings 2023-05-09 08:57:34 +02:00
Chocobozzz
c50bb3c561 Remove unmaintained help script 2023-05-09 08:57:34 +02:00