Commit Graph

3124 Commits

Author SHA1 Message Date
Simon Sawicki
3121512228
[core] Change how Cookie headers are handled
Cookies are now saved and loaded under `cookies` key in the info dict
instead of `http_headers.Cookie`. Cookies passed in headers are
auto-scoped to the input URLs with a warning.

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: Grub4K
2023-07-06 23:14:39 +05:30
coletdjnz
f8b4bcc0a7
[core] Prevent Cookie leaks on HTTP redirect
Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: coletdjnz
2023-07-06 23:14:39 +05:30
bashonly
1ceb657bdd
[fd/external] Scope cookies
- ffmpeg: Calculate cookies from cookiejar and pass with `-cookies` arg instead of `-headers`
- aria2c, curl, wget: Write cookiejar to file and use external FD built-in cookiejar support
- httpie: Calculate cookies from cookiejar instead of `http_headers`
- axel: Calculate cookies from cookiejar and disable http redirection if cookies are passed
    - May break redirects, but axel simply don't have proper cookie support

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: bashonly, coletdjnz
2023-07-06 23:14:38 +05:30
pukkandan
ad8902f616
[ie/vidlii] Handle relative URLs
Closes #7480
2023-07-06 21:40:09 +05:30
pukkandan
94ed638a43
[ie/youtube] Avoid false DRM detection (#7396)
Some master manifests contain a mix of DRM and non-DRM formats
2023-07-06 21:40:07 +05:30
pukkandan
bc344cd456
[core] Allow extractors to mark formats as potentially DRM (#7396)
This is useful for HLS where detecting whether the format is
actually DRM requires the child manifest to be downloaded.

Makes the error message when using `--test` inconsistent,
but doesn't really matter.
2023-07-06 21:40:01 +05:30
pukkandan
906c0bdcd8
[formats] Fix best fallback for storyboards
Partial fix for #7478
2023-07-06 21:39:58 +05:30
pukkandan
337734d4a8
[cleanup] Misc 2023-07-06 21:39:55 +05:30
pukkandan
47bcd43724
[outtmpl] Pad playlist_index etc even when with internal formatting
Closes #7501
2023-07-06 20:22:03 +05:30
pukkandan
662ef1e910
[downloader/http] Avoid infinite loop when no data is received
Closes #7504
2023-07-06 20:22:00 +05:30
coletdjnz
90db9a3c00
[extractor/youtube:stories] Remove (#7459)
YouTube killed them

https://web.archive.org/web/20230630153050/https://support.google.com/youtube/thread/217640760
2023-07-06 19:02:41 +05:30
bashonly
49296437a8
[extractor/twitter] Fix unauthenticated extraction (#7476)
Closes #7473
Authored by: bashonly
2023-07-05 16:27:36 +00:00
bashonly
1cffd621cb
[extractor/twitter:spaces] Fix extraction (#7512)
Closes #7455
Authored by: bashonly
2023-07-05 03:05:52 +00:00
RfadnjdExt
3b7f5300c5
[extractor/googledrive] Fix source format extraction (#7395)
Closes #7344
Authored by: RfadnjdExt
2023-07-05 02:17:13 +00:00
coletdjnz
4dc4d8473c
[extractor/youtube] Ignore incomplete data for comment threads by default (#7475)
For both `--ignore-errors` and `--ignore-errors only_download`. Pass `--no-ignore-errors` to not ignore.

Closes https://github.com/yt-dlp/yt-dlp/issues/7474

Authored by: coletdjnz
2023-07-03 10:47:10 +00:00
c-basalt
8776349ef6
[extractor/vk] VKPlay, VKPlayLive: Add extractors (#7358)
Closes #7107
Authored by: c-basalt
2023-07-02 19:31:00 +00:00
urectanc
af1fd12f67
[extractor/stacommu] Add extractors (#7432)
Authored by: urectanc
2023-06-30 18:27:07 +00:00
coletdjnz
fcbc9ed760
[extractor/youtube:tab] Support shorts-only playlists (#7425)
Fixes https://github.com/yt-dlp/yt-dlp/issues/7424

Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2023-06-29 23:26:27 +00:00
bashonly
a2be9781fb
[extractor/Douyin] Fix extraction from webpage
Closes #7431
Authored by: bashonly
2023-06-27 16:50:02 -05:00
Xiao Han
8f05fbae2a
[extractor/abc] Fix extraction (#7434)
Closes #6433
Authored by: meliber
2023-06-27 21:16:57 +00:00
Aman Salwan
5b4b92769a
[extractor/crunchyroll:music] Fix _VALID_URL (#7439)
Closes #7419
Authored by: AmanSal1, rdamas

Co-authored-by: Robert Damas <robert.damas@byom.de>
2023-06-27 20:28:23 +00:00
pukkandan
91302ed349
[utils] clean_podcast_url: Handle protocol in redirect URL
Closes #7430
2023-06-26 16:19:49 +05:30
pukkandan
f393bbe724
[extractor/sbs] Python 3.7 compat
Closes #7410
2023-06-26 16:14:20 +05:30
pukkandan
8a8af356e3
[downloader/aria2c] Add --no-conf
Closes #7404
2023-06-26 16:13:31 +05:30
pukkandan
d949c10c45
[extractor/youtube] Process post_live over 2 hours 2023-06-26 07:25:52 +05:30
bashonly
ef8509c300
[extractor/kick] Fix _VALID_URL
Closes #7384
Authored by: bashonly
2023-06-25 17:04:42 -05:00
nnoboa
5e16cf92eb
[extractor/AdultSwim] Extract subtitles from m3u8 (#7421)
Authored by: nnoboa
Closes #6191
2023-06-26 01:52:38 +05:30
bashonly
f0a1ff1181
[extractor/qdance] Add extractor (#7420)
Closes #7385
Authored by: bashonly
2023-06-25 18:13:28 +00:00
pukkandan
58786a10f2
[extractor/youtube] Add extractor-arg formats
Closes #7417
2023-06-25 20:14:37 +05:30
pukkandan
e59e20744e
Bugfix for b4e0d75848 2023-06-22 23:45:53 +05:30
Simon
89bed01374
[extractor/youtube] Fix comments' is_favorited (#7390)
Authored by: bbilly1
Closes #7389
2023-06-22 23:38:42 +05:30
github-actions
de4cf77ec1 Release 2023.06.22
Created by: pukkandan

:ci skip all :ci run dl
2023-06-22 08:09:31 +00:00
pukkandan
812cdfa06c
[cleanup] Misc 2023-06-22 13:31:07 +05:30
pukkandan
cd810afe2a
[extractor/youtube] Improve nsig function name extraction 2023-06-22 13:27:18 +05:30
pukkandan
b4e0d75848
Improve --download-sections
* Support negative time-ranges
* Add `*from-url` to obey time-ranges in URL

Closes #7248
2023-06-22 13:03:07 +05:30
Berkan Teber
71dc18fa29
[extractor/youtube] Improve description parsing performance (#7315)
* The parsing is skipped when not needed
* The regex is improved by simulating atomic groups with lookaheads

Authored by: pukkandan, berkanteber
2023-06-22 12:57:54 +05:30
bashonly
98cb1eda7a
[extractor/rheinmaintv] Add extractor (#7311)
Authored by: barthelmannk

Co-authored-by: barthelmannk <81305638+barthelmannk@users.noreply.github.com>
2023-06-22 05:24:52 +00:00
bashonly
774aa09dd6
[extractor/dplay] GlobalCyclingNetworkPlus: Add extractor (#7360)
* Allows `country` API param to be configured with `--xff`/`geo_bypass_country`

Closes #7324
Authored by: bashonly
2023-06-22 05:16:39 +00:00
rexlambert22
f2ff0f6f19
[extractor/motherless] Add gallery support, fix groups (#7211)
Authored by: rexlambert22
2023-06-22 00:00:54 +00:00
pukkandan
5fd8367496
[extractor] Support multiple _VALID_URLs (#5812)
Authored by: nixxo
2023-06-22 03:19:55 +05:30
pukkandan
0dff8e4d1e
Indicate filesize approximated from tbr better 2023-06-22 01:37:55 +05:30
pukkandan
1e75d97db2
[extractor/youtube] Add ios to default clients used
* IOS is affected neither by 403 or by nsig so helps mitigate them preemptively
* IOS also has higher bit-rate "premium" formats though they are not labeled as such
2023-06-22 01:36:06 +05:30
pukkandan
81ca451480
[extractor/youtube] Workaround 403 for android formats
Ref: https://github.com/TeamNewPipe/NewPipe/issues/9038#issuecomment-1289756816
2023-06-22 00:15:22 +05:30
Roland Hieber
3f756c8c40
[extractor/nebula] Fix extractor (#7156)
Closes #7017
Authored by: Lamieur, rohieb

Co-authored-by: Lam <github@Lam.pl>
2023-06-21 08:29:34 +00:00
bashonly
7f9c6a63b1
[cleanup] Misc
Authored by: bashonly
2023-06-21 03:24:24 -05:00
OverlordQ
db22142f6f
[extractor/dropout] Fix season extraction (#7304)
Authored by: OverlordQ
2023-06-21 07:17:07 +00:00
pukkandan
d7cd97e8d8
Fix bug in db3ad8a676
Closes #7367
2023-06-21 12:13:27 +05:30
github-actions
d1b2156149 Release 2023.06.21
Created by: pukkandan

:ci skip all :ci run dl
2023-06-21 04:02:40 +00:00
pukkandan
42f2d40b47
Update to ytdl-commit-07af47
[YouTube] Improve fix for ae8ba2c
07af47960f
2023-06-21 09:21:23 +05:30
pukkandan
1619ab3e67
Bugfix for ebe1b4e34f 2023-06-21 09:21:22 +05:30
pukkandan
84078a8b38
[core] Fix filepath being copied to underlying format dict
Closes #6536
2023-06-21 09:21:21 +05:30
pukkandan
ad54c9130e
[cleanup] Misc
Closes #6288, Closes #7197, Closes #7265, Closes #7353, Closes #5773
Authored by: mikf, freezboltz, pukkandan
2023-06-21 09:21:20 +05:30
Nicolai Dagestad
db3ad8a676
Add option --netrc-cmd (#6682)
Authored by: NDagestad, pukkandan
Closes #1706
2023-06-21 08:37:42 +05:30
MMM
af7585c824
[extractor/tagesschau] Fix single audio urls (#6626)
Authored by: flashdagger
2023-06-21 08:14:12 +05:30
pukkandan
02948a17d9
[update] Do not restart into versions without --update-to 2023-06-21 06:10:40 +05:30
pukkandan
424f3bf033
[downloader/fragment] Do not sleep between fragments
Closes #6599
2023-06-21 06:10:39 +05:30
pukkandan
ebe1b4e34f
[outtmpl] Fix some minor bugs
Closes #7164
2023-06-21 06:10:39 +05:30
pukkandan
a35af4306d
[utils] strftime_or_none: Handle negative timestamps
Closes #6706
Authored by pukkandan, dirkf
2023-06-21 06:10:39 +05:30
pukkandan
93b39cdbd9
Add --compat-option playlist-match-filter
Closes #6073
2023-06-21 06:10:39 +05:30
pukkandan
97afb093d4
[extractor/youtube] Ignore wrong fps of some formats 2023-06-21 06:10:39 +05:30
pukkandan
2e023649ea
[cookies] Revert compatibility breakage in b38d4c941d 2023-06-21 06:10:38 +05:30
pukkandan
51a07b0dca
[extractor/youtube] Prioritize premium formats
Closes #7283
2023-06-21 06:10:38 +05:30
pukkandan
eedda5252c
[utils] FormatSorter: Improve size and br
Closes #1596

Previously, when some formats have accurate size and some approximate,
the ones with accurate size was always prioritized

For formats with known tbr and unknown vbr/abr, we were setting
(vbr=tbr, abr=0) for sorting to work. This is no longer needed.

Authored by pukkandan, u-spec-png
2023-06-21 06:10:38 +05:30
Mozi
5cc09c004b
[extractor/zaiko] ZaikoETicket: Add extractor (#7347)
Authored by: pzhlkj6612
2023-06-20 04:22:36 +00:00
Vladislav
6f69101dc9
[extractor/yappy] YappyProfile: Add extractor (#7346)
Authored by: 7vlad7
2023-06-19 20:43:35 +00:00
garret
81c8b9bdd9
[extractor/nhk] NhkRadiruLive: Add extractor (#7332)
Authored by: garret1317
2023-06-19 13:25:27 +00:00
pukkandan
01aba2519a
[jsinterp] Fix global object extraction
Closes #7327
2023-06-18 04:11:15 +05:30
pukkandan
13ff780953
[postprocessor] Print newline for --progress-template
Closes #7193
2023-06-17 01:43:09 +05:30
pukkandan
ff9b0e071f
[extractor/youtube] Determine audio language using automatic captions 2023-06-17 01:43:03 +05:30
toomyzoom
0a5d7c39e1
[extractor/iwara] Fix authentication (#7137)
Closes #7035, Closes #7207
Authored by: toomyzoom
2023-06-15 23:23:01 +00:00
TxI5
125ffaa173
[extractor/tv4] Fix extractor (#5649)
Closes #5535
Authored by: TxI5, dirkf
2023-06-15 17:57:25 +00:00
foreignBlade
f9213f8a2d
[extractor/stripchat] Fix extractor (#7306)
Closes #7305
Authored by: foreignBlade
2023-06-15 10:56:26 +00:00
Jeong, Heon
fdd69db389
[extractor/afreecatv] Fix extractor (#6283)
Closes #6133
Authored by: blmarket
2023-06-14 19:01:18 +00:00
Elyse
83465fc410
[extractor/ettutv] Add extractor (#6579)
Closes #6359
Authored by: elyse0
2023-06-14 18:54:06 +00:00
RjY
6daaf21092
[extractor/discogs] Add extractor (#6624)
Authored by: rjy
2023-06-14 18:40:06 +00:00
hoaluvn
7bcd481321
[extractor/urplay] Extract all subtitles (#7309)
Authored by: hoaluvn
2023-06-14 21:22:17 +05:30
bashonly
c8561c6d03
[extractor/wrestleuniverse] Fix cookies support
Closes #7298
Authored by: bashonly
2023-06-13 15:49:18 -05:00
Cyberes
cab94a0cd8
[extractor/funker530] Add extractor (#7291)
Authored by: Cyberes
2023-06-13 03:23:17 +00:00
c-basalt
345b4c0aed
[extractor/zaiko] Add extractor (#7254)
Closes #7196
Authored by: c-basalt
2023-06-12 18:12:09 +00:00
linsui
8790ea7b25
[extractor/ximalaya] Sort playlist entries (#7292)
Authored by: linsui
2023-06-12 13:32:50 +05:30
puc9
ab6057ec80
[extractor/tiktok] Fix resolution extraction (#7237)
Authored by: puc9
2023-06-11 18:57:59 +00:00
bashonly
9d7fde89a4
[extractor/zee5] Fix extraction of new content (#7280)
Authored by: bashonly
2023-06-11 17:15:05 +00:00
bashonly
1a2eb5bda5
[extractor/odnoklassniki] Fix formats extraction (#7217)
Closes #2959, Closes #4462, Closes #7201
Authored by: bashonly
2023-06-11 17:06:34 +00:00
DataGhost
f8ae441501
[extractor/Dumpert] Fix m3u8 and support new URL pattern (#6091)
Authored by: DataGhost, pukkandan
Closes #5032
2023-06-11 20:47:26 +05:30
bashonly
b4a252fba8
[jsinterp] Fix division (#7279)
* Fixes nsig decryption for Youtube JS player `8c7583ff`

Authored by: bashonly
2023-06-10 22:49:12 +00:00
bashonly
4f7b11cc1c
[extractor/voot] Fix extractor (#7227)
Closes #6715
Authored by: bashonly
2023-06-10 20:43:22 +00:00
bashonly
d1795f4a6a
[extractor/twitter] Add login support (#7258)
Closes #6951
Authored by: bashonly
2023-06-08 18:47:13 +00:00
bashonly
44c0d66442
[extractor/lbry] Extract original quality formats (#7257)
Closes #7251
Authored by: bashonly
2023-06-08 18:36:09 +00:00
coletdjnz
8213ce28a4
[extractor/youtube] Extract channel_is_verified (#7213)
Authored by: coletdjnz
2023-06-08 19:15:39 +05:30
pukkandan
14a14335b2
[extractor/youtube] Misc cleanup
Authored by: coletdjnz
2023-06-08 19:14:57 +05:30
stanoarn
c2b801fea5
[extractor/rozhlas] MujRozhlas: Add extractor (#7129)
Authored by: stanoarn
2023-06-07 20:18:06 +00:00
bashonly
59d9fe0831
[extractor/mgtv] Fix formats extraction (#7234)
Closes #7008
Authored by: bashonly
2023-06-05 15:52:45 +00:00
bashonly
ee0ed0338d
[extractor/zdf] Fix formats extraction
Closes #7238, Closes #7240
Authored by: bashonly
2023-06-05 10:40:48 -05:00
bashonly
c2a1bdb009
[extractor/tiktok] Extract 1080p adaptive formats (#7228)
Closes #7109
Authored by: bashonly
2023-06-04 14:28:40 +00:00
bashonly
7f8ddebbb5
[extractor/hotstar] Support /shows/ URLs (#7225)
Closes #6463
Authored by: bashonly
2023-06-04 14:19:16 +00:00
bashonly
7bc9251746
[extractor/shemaroome] Pass stream_key header to downloader (#7224)
Closes #7133
Authored by: bashonly
2023-06-04 14:07:13 +00:00
bashonly
4815d35c19
[extractor/sonyliv] Fix login with token (#7223)
Authored by: bashonly
2023-06-04 13:49:10 +00:00
bashonly
97d60ad8cd
[extractor/foxnews] Fix extractors (#7222)
Closes #6050
Authored by: bashonly
2023-06-04 13:37:59 +00:00
bashonly
5ee9a7d6e1
[extractor/sverigesradio] Support slug URLs (#7220)
Closes #7145
Authored by: bashonly
2023-06-04 12:15:09 +00:00
bashonly
971d901d12
[extractor/tencent] Fix fatal metadata extraction (#7219)
Closes #7177
Authored by: bashonly
2023-06-04 12:03:44 +00:00