mirror/slsk-batchdl
1
0
Fork 0
mirror of https://github.com/fiso64/slsk-batchdl.git synced 2024-12-22 14:32:40 +00:00
Code Issues Activity

stuff

Browse source
This commit is contained in:
fiso64 2023-06-12 17:04:49 +02:00
parent 134e8ce442
commit b4a715d43e
5 changed files with 1378 additions and 772 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

188
README.md
View file

@ -2,89 +2,125 @@
A batch downloader for Soulseek using Soulseek.NET. Accepts CSV files, Spotify & YouTube urls.
- Download tracks from a csv file:
```
Usage: slsk-batchdl.exe [OPTIONS]
Options:
-p --parent <path> Downloaded music will be placed here
-n --name <name> Folder / playlist name. If not specified, the name of the csv file / spotify / yt playlist is used.
--username <username> Soulseek username
--password <password> Soulseek password
--spotify <url> Download a spotify playlist. "likes" to download all your liked music.
--spotify-id <id> Your spotify client id (use if the default fails or if playlist private)
--spotify-secret <sec> Your spotify client secret (use if the default fails or if playlist private)
--youtube <url> Get tracks from a YouTube playlist
--youtube-key <key> Provide an API key if you also want to search for unavailable uploads
--no-channel-search Enable to also perform a search without channel name if nothing was found (only for yt).
--csv <path> Use a csv file containing track info to download
--artist-col <column> Artist or uploader name column
--title-col <column> Title or track name column
--album-col <column> CSV album column name. Optional, may improve searching, slower
--length-col <column> CSV duration column name. Recommended, will improve accuracy
--time-unit <unit> Time unit for the track duration column, ms or s (default: s)
--yt-desc-col <column> Description column name. Use with --yt-parse.
--yt-id-col <column> Youtube video ID column (only needed if length-col or yt-desc-col don't exist). Use with --yt-parse.
--yt-parse Enable if you have a csv file of YouTube video titles and channel names; attempt to parse.
-s --single <str> Search & download a specific track
-a --album <str> Does nothing
--pref-format <format> Preferred file format (default: mp3)
--pref-length-tol <tol> Preferred length tolerance (if length col provided) (default: 3)
--pref-min-bitrate <rate> Preferred minimum bitrate (default: 200)
--pref-max-bitrate <rate> Preferred maximum bitrate (default: 2200)
--pref-max-samplerate <rate> Preferred maximum sample rate (default: 96000)
--pref-danger-words <list> Comma separated list of words that must appear in either both search result and track title, or in neither of the two. Case-insensitive. (default: "mix, edit,dj ,cover")
--nec-format <format> Necessary file format
--nec-length-tolerance <tol> Necessary length tolerance (default: 3)
--nec-min-bitrate <rate> Necessary minimum bitrate
--nec-max-bitrate <rate> Necessary maximum bitrate
--nec-max-samplerate <rate> Necessary maximum sample rate
--nec-danger-words <list> Comma separated list of words that must appear in either both search result and track title, or in neither of the two. Case-insensitive. (default: "mix, edit,dj ,cover")
--album-search Also search for "[Album name] [track name]". Occasionally helps to find more, slower.
--no-diacr-search Also perform a search without diacritics
--skip-existing Skip if a track matching the conditions is found in the output folder or your music library (if provided)
--skip-notfound Skip searching for tracks that weren't found in Soulseek last time
--remove-ft Remove "ft." or "feat." and everything after from the track names.
--remove-strings <strings> Comma separated list of strings to remove when searching for tracks. Case insesitive.
--music-dir <path> Specify to also skip downloading tracks which are in your library, use with --skip-existing
--reverse Download tracks in reverse order
--skip-if-pref-failed Skip if preferred versions of a track exist but failed to download. If no pref. versions were found, download as normal.
--create-m3u Create an m3u playlist file
--m3u-only Only create an m3u playlist file with existing tracks and exit
--m3u <path> Where to place created m3u files (--parent by default)
--yt-dlp Use yt-dlp to download tracks that weren't found on Soulseek. yt-dlp must be available from the command line.
--yt-dlp-f <format> yt-dlp audio format (default: "bestaudio/best")
--search-timeout <ms> Maximal search time (default: 10000)
--max-stale-time <ms> Maximal download time with no progress (default: 60000)
--concurrent-processes <num> Max concurrent searches / downloads (default: 2)
--max-retries <num> Maximum number of users to try downloading from before skipping track (default: 30)
--slow-output Enable if the progress bars aren't properly updated (bug)
slsk-batchdl --csv test.csv --artist-col "Artist Name(s)" --track-col "Track Name" --length-col "Duration (ms)" --time-unit ms
```
Files satisfying `pref` conditions will be preferred. Files not satisfying `nec` conditions will not be downloaded.
You can omit the column names provided they are named predictably (like in this example). Use `--print-tracks` before downloading to check if everything has been parsed correctly.
Download tracks from a csv file and create m3u:
- Download spotify likes while skipping existing songs, and create an m3u file:
```
slsk-batchdl.exe -p "C:\Users\fiso64\Music\Playlists" --csv "C:\Users\fiso64\Downloads\test.csv" --username "fakename" --password "fakepass" --artist-col "Artist Name(s)" --track-col "Track Name" --length-col "Duration (ms)" --time-unit "ms" --skip-existing --create-m3u --pref-format "flac"
slsk-batchdl --spotify likes --m3u --skip-existing
```
You might need to provide an id and secret when using spotify (e.g when downloading a private playlist), which you can get here https://developer.spotify.com/dashboard/applications. Create an app, then select it and add `http://localhost:48721/callback` as a redirect url in the settings.
Download spotify playlist with fallback to yt-dlp and create a m3u:
- Download the first 10 songs of a youtube playlist:
```
slsk-batchdl.exe --spotify <url> -p "C:\Users\fiso64\Music\Playlists" --m3u "C:\Users\fiso64\Documents\MusicBee\Playlists" --music-dir "C:\Users\fiso64\Music" --username "fakename" --password "fakepass" --skip-existing --pref-format "flac" --yt-dlp
```
You might need to provide an id and secret when using spotify, which you can get here https://developer.spotify.com/dashboard/applications. Create an app, then select it and add `http://localhost:48721/callback` as a redirect url in the settings.
Download youtube playlist:
```
--youtube "https://www.youtube.com/playlist?list=PLI_eFW8NAFzYAXZ5DrU6E6mQ_XfhaLBUX" -p "C:\Users\fiso64\Music\Playlists" --username "fakename" --password "fakepass"
slsk-batchdl -n 10 --youtube "https://www.youtube.com/playlist?list=PLI_eFW8NAFzYAXZ5DrU6E6mQ_XfhaLBUX"
```
To include unavailable videos, you will need to provide an api key with `--youtube-key`. Get it here https://console.cloud.google.com. Create a new project, click "Enable Api" and search for "youtube data", then follow the prompts.
- Search & download a specific song, preferring flac and wav files:
```
slsk-batchdl "title=MC MENTAL @ HIS BEST,duration=242" --pref-format "flac,wav"
```
### Options:
```
Usage: slsk-batchdl [OPTIONS]
Options:
--user <username> Soulseek username
--pass <password> Soulseek password
--spotify <url> Download a spotify playlist ("likes" for liked music)
--spotify-id <id> Your spotify client id (required for private playlists)
--spotify-secret <sec> Your spotify client secret (required for private playlists)
--youtube <url> Get tracks from a YouTube playlist
--youtube-key <key> Provide an API key to include unavailable uploads
--csv <path> Use a csv file containing track info to download
--artist-col <column> Artist or uploader column name
--title-col <column> Title or track name column name
--album-col <column> Track album column name (optional for more results)
--length-col <column> Track duration column name (optional for better accuracy)
--time-unit <unit> Time unit in track duration column, ms or s (default: s)
--yt-desc-col <column> YT description column name (optional, use with --yt-parse)
--yt-id-col <column> Youtube video ID column (optional, use with --yt-parse)
--yt-parse Enable if you have a csv file of YouTube video titles and
channel names; attempt to parse them into title and artist
-s --single <str> Search & download a specific track. <str> is a simple
search string, or a comma-separated list of properties:
"title=Song Name,artist=Artist Name,length=215"
-p --path <path> Place downloaded files in custom path
-f --folder <name> Custom folder name (default: provided playlist name)
-n --number <maxtracks> Download at most n tracks of a playlist
-o --offset <offset> Skip a specified number of tracks
--reverse Download tracks in reverse order
--name-format <format> Name format for downloaded tracks, e.g "{artist} - {title}"
--m3u Create an m3u8 playlist file
--pref-format <format> Preferred file format(s), comma-separated (default: mp3)
--pref-length-tol <tol> Preferred length tolerance in seconds (default: 3)
--pref-min-bitrate <rate> Preferred minimum bitrate (default: 200)
--pref-max-bitrate <rate> Preferred maximum bitrate (default: 2200)
--pref-max-samplerate <rate> Preferred maximum sample rate (default: 96000)
--pref-strict-title Prefer download if filename contains track title
--pref-strict-artist Prefer download if filepath contains track artist
--pref-danger-words <list> Comma-separated list of words that must appear in either
both search result and track title, or in neither of the
two, case-insensitive (default:"mix, edit, dj, cover")
--nec-format <format> Necessary file format(s), comma-separated
--nec-length-tol <tol> Necessary length tolerance in seconds (default: 3)
--nec-min-bitrate <rate> Necessary minimum bitrate
--nec-max-bitrate <rate> Necessary maximum bitrate
--nec-max-samplerate <rate> Necessary maximum sample rate
--nec-strict-title Only download if filename contains track title
--nec-strict-artist Only download if filepath contains track artist
--nec-danger-words <list> Comma-separated list of words that must appear in either
both search result and track title, or in neither of the
two. Case-insensitive. (default:"mix, edit, dj, cover")
--skip-existing Skip if a track matching nec. conditions is found in the
output folder or your music library (if provided)
--skip-mode <mode> "name": Use only filenames to check if a track exists
"name-precise": Use filenames and check nec-cond (default)
"tag": Use tags (slower)
"tag-precise": Use tags and check all nec. cond. (slower)
--music-dir <path> Specify to skip downloading tracks found in a music library
Use with --skip-existing
--skip-not-found Skip searching for tracks that weren't found on Soulseek
last run
--remove-ft Remove "ft." or "feat." and everything after from the
track names before searching.
--album-search Also search for album name before filtering for track name.
Sometimes helps to find more, but slower.
--artist-search Also search for artist, before filtering for track name.
Sometimes helps to find more, but slower.
--no-artist-search Also perform a search without artist name if nothing was
found. Only use if the source is imprecise
and the provided "artist" is possibly wrong (yt, sc)
--no-regex-search <reg> Also perform a search with a regex pattern removed from the
titles and artist names
--no-diacr-search Also perform a search without diacritics
-d --desperate Equivalent to enabling all additional searches
--yt-dlp Use yt-dlp to download tracks that weren't found on
Soulseek. yt-dlp must be available from the command line.
--search-timeout <ms> Maximal search time (ms, default: 6000)
--max-stale-time <ms> Maximal download time with no progress (ms, default: 50000)
--concurrent-processes <num> Max concurrent searches & downloads (default: 2)
--display <str> "single" (default): Show transfer state and percentage.
"double": Also show a progress bar. "simple": simple
--print-tracks Do not search, only print all tracks to be downloaded
--print-results Do not download, print search results satisfying nec. cond.
--print-results-full Do not download, print all search results with full path
```
Files satisfying `pref-` conditions will be preferred. Files not satisfying `nec-` conditions will not be downloaded. For example, `--nec-length-tol` is set to 3 by default, which means that files whose duration differs from the supplied duration by more than 3 seconds will not be downloaded. Increase it to download e.g a youtube playlist of music videos with intros/outros, or disable it entirely by setting it to 99999.
Supports .conf files: Create a file named `slsk-batchdl.conf` in the same directory as the exe and write your arguments there, e.g:
```
--username "fakename"
@ -92,8 +128,6 @@ Supports .conf files: Create a file named `slsk-batchdl.conf` in the same direct
--pref-format "flac"
```
## Notes:
- YouTube playlist downloading is unreliable since there are no track name / artist tags
### Notes:
- The CSV file must be saved with `,` as field delimiter and `"` as string delimiter, encoded with UTF8
- 40% of the code was written by ChatGPT
- Sometimes it starts vomiting text in the terminal (I don't know why), use --slow-output if that's an issue.
- `--display single` and especially `double` can cause the printed lines to be duplicated or overwritten on some configurations. Use `simple` if that's an issue. In my testing on Windows, the terminal app seems to be affected by this (unlike the old command prompt).

1815
slsk-batchdl/Program.cs
View file

File diff suppressed because it is too large Load diff

20
slsk-batchdl/Spotify.cs
View file

@ -71,14 +71,13 @@ public class Spotify
return true;
}
public async Task<List<Track>> GetLikes(StringEdit stringEdit)
public async Task<List<Track>> GetLikes(int max = int.MaxValue, int offset = 0)
{
if (!loggedIn)
throw new Exception("Can't get liked music, not logged in");
List<Track> res = new List<Track>();
int offset = 0;
int limit = 50;
int limit = Math.Min(max, 50);
while (true)
{
@ -88,30 +87,30 @@ public class Spotify
{
string[] artists = ((IEnumerable<object>)track.Track.ReadProperty("artists")).Select(a => (string)a.ReadProperty("name")).ToArray();
string artist = artists[0];
string name = stringEdit.Edit((string)track.Track.ReadProperty("name"));
string name = (string)track.Track.ReadProperty("name");
string album = (string)track.Track.ReadProperty("album").ReadProperty("name");
int duration = (int)track.Track.ReadProperty("durationMs");
res.Add(new Track { Album = album, ArtistName = artist, TrackTitle = name, Length = duration / 1000 });
}
if (tracks.Items.Count < limit)
if (tracks.Items.Count < limit || res.Count >= max)
break;
offset += limit;
limit = Math.Min(max - res.Count, 50);
}
return res;
}
public async Task<(string?, List<Track>)> GetPlaylist(string url, StringEdit stringEdit)
public async Task<(string?, List<Track>)> GetPlaylist(string url, int max = int.MaxValue, int offset = 0)
{
var playlistId = GetPlaylistIdFromUrl(url);
var p = await _client.Playlists.Get(playlistId);
List<Track> res = new List<Track>();
int offset = 0;
int limit = 100;
int limit = Math.Min(max, 100);
while (true)
{
@ -121,16 +120,17 @@ public class Spotify
{
string[] artists = ((IEnumerable<object>)track.Track.ReadProperty("artists")).Select(a => (string)a.ReadProperty("name")).ToArray();
string artist = artists[0];
string name = stringEdit.Edit((string)track.Track.ReadProperty("name"));
string name = (string)track.Track.ReadProperty("name");
string album = (string)track.Track.ReadProperty("album").ReadProperty("name");
int duration = (int)track.Track.ReadProperty("durationMs");
res.Add(new Track { Album = album, ArtistName = artist, TrackTitle = name, Length = duration / 1000 });
}
if (tracks.Items.Count < limit)
if (tracks.Items.Count < limit || res.Count >= max)
break;
offset += limit;
limit = Math.Min(max - res.Count, 100);
}
return (p.Name, res);

99
slsk-batchdl/YouTube.cs
View file

@ -11,7 +11,7 @@ public static class YouTube
private static YouTubeService? youtubeService = null;
public static string apiKey = "";
public static async Task<(string, List<Track>)> GetTracksApi(string url, StringEdit strEdit)
public static async Task<(string, List<Track>)> GetTracksApi(string url, int max = int.MaxValue, int offset = 0)
{
StartService();
@ -25,38 +25,28 @@ public static class YouTube
var playlistItemsRequest = youtubeService.PlaylistItems.List("snippet,contentDetails");
playlistItemsRequest.PlaylistId = playlistId;
playlistItemsRequest.MaxResults = 100;
playlistItemsRequest.MaxResults = Math.Min(max, 100);
var tracksDict = await GetDictYtExplode(url, strEdit);
var tracksDict = await GetDictYtExplode(url, max, offset);
var tracks = new List<Track>();
int count = 0;
while (playlistItemsRequest != null)
while (playlistItemsRequest != null && count < max + offset)
{
var playlistItemsResponse = playlistItemsRequest.Execute();
foreach (var playlistItem in playlistItemsResponse.Items)
{
if (tracksDict.ContainsKey(playlistItem.Snippet.ResourceId.VideoId))
if (count >= offset)
{
tracks.Add(tracksDict[playlistItem.Snippet.ResourceId.VideoId]);
}
else
{
var title = "";
var uploader = "";
var length = 0;
var desc = "";
if (tracksDict.ContainsKey(playlistItem.Snippet.ResourceId.VideoId))
tracks.Add(tracksDict[playlistItem.Snippet.ResourceId.VideoId]);
else
{
var title = "";
var uploader = "";
var length = 0;
var desc = "";
try
{
var video = await youtube.Videos.GetAsync(playlistItem.Snippet.ResourceId.VideoId);
title = video.Title;
uploader = video.Author.Title;
length = (int)video.Duration.Value.TotalSeconds;
desc = video.Description;
}
catch
{
var videoRequest = youtubeService.Videos.List("contentDetails,snippet");
videoRequest.Id = playlistItem.Snippet.ResourceId.VideoId;
var videoResponse = videoRequest.Execute();
@ -67,11 +57,14 @@ public static class YouTube
uploader = videoResponse.Items[0].Snippet.ChannelTitle;
length = (int)XmlConvert.ToTimeSpan(videoResponse.Items[0].ContentDetails.Duration).TotalSeconds;
desc = videoResponse.Items[0].Snippet.Description;
}
Track track = await ParseTrackInfo(strEdit.Edit(title), uploader, playlistItem.Snippet.ResourceId.VideoId, length, false, desc);
tracks.Add(track);
Track track = await ParseTrackInfo(title, uploader, playlistItem.Snippet.ResourceId.VideoId, length, false, desc);
tracks.Add(track);
}
}
if (++count >= max + offset)
break;
}
if (tracksDict.Count >= 200)
@ -81,10 +74,10 @@ public static class YouTube
}
playlistItemsRequest.PageToken = playlistItemsResponse.NextPageToken;
if (playlistItemsRequest.PageToken == null)
{
if (playlistItemsRequest.PageToken == null || count >= max + offset)
playlistItemsRequest = null;
}
else
playlistItemsRequest.MaxResults = Math.Min(offset + max - count, 100);
}
Console.WriteLine();
@ -236,55 +229,69 @@ public static class YouTube
public static void StopService()
{
//try { youtubeService.Dispose(); }
//catch { }
youtubeService = null;
}
public static async Task<Dictionary<string, Track>> GetDictYtExplode(string url, StringEdit strEdit)
public static async Task<Dictionary<string, Track>> GetDictYtExplode(string url, int max = int.MaxValue, int offset = 0)
{
var youtube = new YoutubeClient();
var playlist = await youtube.Playlists.GetAsync(url);
var tracks = new Dictionary<string, Track>();
int count = 0;
await foreach (var video in youtube.Playlists.GetVideosAsync(playlist.Id))
{
var title = strEdit.Edit(video.Title);
var uploader = video.Author.Title;
var ytId = video.Id.Value;
var length = (int)video.Duration.Value.TotalSeconds;
if (count >= offset && count < offset + max)
{
var title = video.Title;
var uploader = video.Author.Title;
var ytId = video.Id.Value;
var length = (int)video.Duration.Value.TotalSeconds;
var track = await ParseTrackInfo(title, uploader, ytId, length, true);
var track = await ParseTrackInfo(title, uploader, ytId, length, true);
tracks[ytId] = track;
tracks[ytId] = track;
}
if (count++ >= offset + max)
break;
}
return tracks;
}
public static async Task<(string, List<Track>)> GetTracksYtExplode(string url, StringEdit strEdit)
public static async Task<(string, List<Track>)> GetTracksYtExplode(string url, int max = int.MaxValue, int offset = 0)
{
var youtube = new YoutubeClient();
var playlist = await youtube.Playlists.GetAsync(url);
var playlistTitle = playlist.Title;
var tracks = new List<Track>();
int count = 0;
await foreach (var video in youtube.Playlists.GetVideosAsync(playlist.Id))
{
var title = strEdit.Edit(video.Title);
var uploader = video.Author.Title;
var ytId = video.Id.Value;
var length = (int)video.Duration.Value.TotalSeconds;
if (count >= offset && count < offset + max)
{
var title = video.Title;
var uploader = video.Author.Title;
var ytId = video.Id.Value;
var length = (int)video.Duration.Value.TotalSeconds;
var track = await ParseTrackInfo(title, uploader, ytId, length, true);
var track = await ParseTrackInfo(title, uploader, ytId, length, true);
tracks.Add(track);
tracks.Add(track);
}
if (count++ >= offset + max)
break;
}
return (playlistTitle, tracks);
}
public static async Task<string> UrlToId(string url)
{
var playlist = await youtube.Playlists.GetAsync(url);

14
slsk-batchdl/slsk-batchdl.csproj
View file

@ -7,8 +7,20 @@
<Nullable>enable</Nullable>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
<DefineConstants>$(DefineConstants)TRACE;</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
<DefineConstants>$(DefineConstants)TRACE;</DefineConstants>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Goblinfactory.Konsole" Version="6.2.2" />
<Compile Remove="Test.cs" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Goblinfactory.ProgressBar" Version="1.0.0" />
<PackageReference Include="Google.Apis.YouTube.v3" Version="1.60.0.2945" />
<PackageReference Include="Soulseek" Version="6.1.1" />
<PackageReference Include="SpotifyAPI.Web" Version="7.0.0" />