mirror/slsk-batchdl
1
0
Fork 0
mirror of https://github.com/fiso64/slsk-batchdl.git synced 2024-12-22 22:42:41 +00:00
Code Issues Activity

stuff

Browse source
This commit is contained in:
fiso64 2023-11-28 14:31:42 +01:00
parent 05d8accf93
commit 20d8310d20
3 changed files with 666 additions and 399 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

37
README.md
View file

@ -4,37 +4,37 @@ A batch downloader for Soulseek using Soulseek.NET. Accepts CSV files and Spotif
#### Download tracks from a csv file: #### Download tracks from a csv file:
``` ```
slsk-batchdl -i test.csv slsk-batchdl test.csv
``` ```
Use `--print tracks` before downloading to check if everything has been parsed correctly. The names of the columns should be: `Artist`, `Title`, `Album`, `Length`. Only the title column is required, but any additional info improves search. Use `--print tracks` before downloading to check if everything has been parsed correctly. The names of the columns should be: `Artist`, `Title`, `Album`, `Length`. Only the title column is required, but any additional info improves search.
#### Download spotify likes while skipping existing songs: #### Download spotify likes while skipping existing songs:
``` ```
slsk-batchdl -i spotify-likes --skip-existing slsk-batchdl spotify-likes --skip-existing
``` ```
To download private playlists or liked songs you will need to provide a client id and secret, which you can get here https://developer.spotify.com/dashboard/applications. Create an app and add `http://localhost:48721/callback` as a redirect url in its settings. To download private playlists or liked songs you will need to provide a client id and secret, which you can get here https://developer.spotify.com/dashboard/applications. Create an app and add `http://localhost:48721/callback` as a redirect url in its settings.
#### Download youtube playlist (with fallback to yt-dlp), including deleted videos: #### Download youtube playlist (with fallback to yt-dlp), including deleted videos:
``` ```
slsk-batchdl --get-deleted --yt-dlp -i "https://www.youtube.com/playlist?list=PLI_eFW8NAFzYAXZ5DrU6E6mQ_XfhaLBUX" slsk-batchdl --get-deleted --yt-dlp "https://www.youtube.com/playlist?list=PLI_eFW8NAFzYAXZ5DrU6E6mQ_XfhaLBUX"
``` ```
Playlists are retrieved using the YoutubeExplode library which unfortunately doesn't always return all videos. You can use the official API by providing a key with `--youtube-key`. Get it here https://console.cloud.google.com. Create a new project, click "Enable Api" and search for "youtube data", then follow the prompts. Playlists are retrieved using the YoutubeExplode library which unfortunately doesn't always return all videos. You can use the official API by providing a key with `--youtube-key`. Get it here https://console.cloud.google.com. Create a new project, click "Enable Api" and search for "youtube data", then follow the prompts.
#### Search & download a specific song: #### Search & download a specific song:
``` ```
slsk-batchdl -i "title=MC MENTAL @ HIS BEST,length=242" --pref-format "flac,wav" slsk-batchdl "title=MC MENTAL @ HIS BEST,length=242" --pref-format "flac,wav"
``` ```
#### Find an artist's songs which aren't in your library: #### Find an artist's songs which aren't in your library:
``` ```
slsk-batchdl -i "artist=MC MENTAL" --aggregate --print tracks --skip-existing --music-dir "path\to\music" slsk-batchdl "artist=MC MENTAL" --aggregate --print tracks --skip-existing --music-dir "path\to\music"
``` ```
### Options: ### Options:
``` ```
Usage: slsk-batchdl -i <input> [OPTIONS] Usage: slsk-batchdl <input> [OPTIONS]
-i --input <input> <input> is one of the following: <input> <input> is one of the following:
Spotify playlist url or "spotify-likes": Download a spotify Spotify playlist url or "spotify-likes": Download a spotify
playlist or your liked songs. --spotify-id and playlist or your liked songs. --spotify-id and
@ -84,12 +84,12 @@ Options:
-n --number <maxtracks> Download the first n tracks of a playlist -n --number <maxtracks> Download the first n tracks of a playlist
-o --offset <offset> Skip a specified number of tracks -o --offset <offset> Skip a specified number of tracks
--reverse Download tracks in reverse order --reverse Download tracks in reverse order
--remove-from-playlist Remove downloaded tracks from playlist (for spotify only) --remove-from-playlist Remove downloaded tracks from playlist (spotify only)
--name-format <format> Name format for downloaded tracks, e.g "{artist} - {title}" --name-format <format> Name format for downloaded tracks, e.g "{artist} - {title}"
--m3u Create an m3u8 playlist file --m3u Create an m3u8 playlist file
--format <format> Accepted file format(s), comma-separated --format <format> Accepted file format(s), comma-separated
--length-tol <tol> Length tolerance in seconds (default: 3) --length-tol <sec> Length tolerance in seconds (default: 3)
--min-bitrate <rate> Minimum file bitrate --min-bitrate <rate> Minimum file bitrate
--max-bitrate <rate> Maximum file bitrate --max-bitrate <rate> Maximum file bitrate
--max-samplerate <rate> Maximum file sample rate --max-samplerate <rate> Maximum file sample rate
@ -100,7 +100,7 @@ Options:
both search result and track title or in neither of the both search result and track title or in neither of the
two. Case-insensitive. (default:"remix, edit,cover") two. Case-insensitive. (default:"remix, edit,cover")
--pref-format <format> Preferred file format(s), comma-separated (default: mp3) --pref-format <format> Preferred file format(s), comma-separated (default: mp3)
--pref-length-tol <tol> Preferred length tolerance in seconds (default: 2) --pref-length-tol <sec> Preferred length tolerance in seconds (default: 2)
--pref-min-bitrate <rate> Preferred minimum bitrate (default: 200) --pref-min-bitrate <rate> Preferred minimum bitrate (default: 200)
--pref-max-bitrate <rate> Preferred maximum bitrate (default: 2200) --pref-max-bitrate <rate> Preferred maximum bitrate (default: 2200)
--pref-max-samplerate <rate> Preferred maximum sample rate (default: 96000) --pref-max-samplerate <rate> Preferred maximum sample rate (default: 96000)
@ -108,7 +108,7 @@ Options:
--pref-banned-users <list> Comma-separated list of users to deprioritize --pref-banned-users <list> Comma-separated list of users to deprioritize
--pref-danger-words <list> Comma-separated list of words that should appear in either --pref-danger-words <list> Comma-separated list of words that should appear in either
both search result and track title or in neither of the both search result and track title or in neither of the
two. (default: see github) two. (default: "mix,dj , edit,cover,(")
-s --skip-existing Skip if a track matching file conditions is found in the -s --skip-existing Skip if a track matching file conditions is found in the
output folder or your music library (if provided) output folder or your music library (if provided)
@ -122,20 +122,28 @@ Options:
during the last run. during the last run.
--remove-ft Remove "ft." or "feat." and everything after from the --remove-ft Remove "ft." or "feat." and everything after from the
track names before searching track names before searching
--remove-regex <regex> Remove a regex from all track names and artist names --remove-regex <regex> Remove a regex from all track titles and artist names
--no-artist-search Perform a search without artist name if nothing was --no-artist-search Perform a search without artist name if nothing was
found. Only use for sources such as youtube or soundcloud found. Only use for sources such as youtube or soundcloud
where the "artist" could just be an uploader. where the "artist" could just be an uploader.
--artist-search Also try to find track by searching for the artist only --artist-search Also try to find track by searching for the artist only
--no-diacr-search Also perform a search without diacritics --no-diacr-search Also perform a search without diacritics
--no-regex-search <regex> Also perform a search without a regex pattern --no-regex-search <regex> Also perform a search without a regex pattern
--levenshtein-weight <num> Results are sorted by the distance between the filename
and track title times the weight (among other things). 1
means each differing character will downrank the result, 0
disables this part of the sorting algorithm. (default: 0.5)
--yt-dlp Use yt-dlp to download tracks that weren't found on --yt-dlp Use yt-dlp to download tracks that weren't found on
Soulseek. yt-dlp must be available from the command line. Soulseek. yt-dlp must be available from the command line.
--config <path> Specify config file location --config <path> Manually specify config file location
--search-timeout <ms> Max search time in ms (default: 6000) --search-timeout <ms> Max search time in ms (default: 6000)
--max-stale-time <ms> Max download time without progress in ms (default: 50000) --max-stale-time <ms> Max download time without progress in ms (default: 50000)
--concurrent-downloads <num> Max concurrent searches & downloads (default: 2) --concurrent-downloads <num> Max concurrent downloads (default: 2)
--searches-per-time <num> Max searches per time interval. Higher values may cause
30-minute bans. (default: 34)
--searches-time <sec> Controls how often available searches are replenished.
Lower values may cause 30-minute bans. (default: 220)
--display <option> Changes how searches and downloads are displayed: --display <option> Changes how searches and downloads are displayed:
single (default): Show transfer state and percentage single (default): Show transfer state and percentage
double: Transfer state and a large progress bar double: Transfer state and a large progress bar
@ -160,3 +168,4 @@ Configuration files: Create a file named `slsk-batchdl.conf` in the same directo
### Notes: ### Notes:
- The CSV file must use `"` as string delimiter and be encoded with UTF8 - The CSV file must use `"` as string delimiter and be encoded with UTF8
- `--display single` and especially `double` can cause the printed lines to be duplicated or overwritten on some configurations. Use `simple` if that's an issue. - `--display single` and especially `double` can cause the printed lines to be duplicated or overwritten on some configurations. Use `simple` if that's an issue.
- The server will ban you for 30 minutes if too many searches are performed within a short timespan. Adjust `--searches-per-time` and `--searches-time` in case it happens. By default it's configured to allow up to 34 searches every 220 seconds. These values were determined through experimentation as unfortunately I couldn't find any information regarding soulseek's rate limits, so they may be incorrect. You can also use `--random-login` to re-login with a random username and password automatically.

426
slsk-batchdl/Program.cs
View file

@ -1,9 +1,11 @@
using AngleSharp.Dom; using AngleSharp.Css;
using AngleSharp.Dom;
using Konsole; using Konsole;
using Newtonsoft.Json.Linq; using Newtonsoft.Json.Linq;
using Soulseek; using Soulseek;
using System; using System;
using System.Collections.Concurrent; using System.Collections.Concurrent;
using System.Data;
using System.Diagnostics; using System.Diagnostics;
using System.Diagnostics.CodeAnalysis; using System.Diagnostics.CodeAnalysis;
using System.IO; using System.IO;
@ -12,11 +14,15 @@ using System.Net.NetworkInformation;
using System.Net.Sockets; using System.Net.Sockets;
using System.Text; using System.Text;
using System.Text.RegularExpressions; using System.Text.RegularExpressions;
using System.Threading;
using System.Xml.Linq; using System.Xml.Linq;
using TagLib.Id3v2; using TagLib.Id3v2;
using TagLib.Matroska; using TagLib.Matroska;
using YoutubeExplode.Playlists; using YoutubeExplode.Playlists;
using ProgressBar = Konsole.ProgressBar;
static class Program static class Program
{ {
static SoulseekClient client = new SoulseekClient(); static SoulseekClient client = new SoulseekClient();
@ -36,7 +42,7 @@ static class Program
MaxSampleRate = 96000, MaxSampleRate = 96000,
StrictTitle = true, StrictTitle = true,
StrictArtist = false, StrictArtist = false,
DangerWords = new string[] { "mix", "dj ", " edit", "cover" }, DangerWords = new string[] { "mix", " edit", "cover", "karaoke" },
BannedUsers = { }, BannedUsers = { },
AcceptNoLength = false, AcceptNoLength = false,
}; };
@ -49,7 +55,7 @@ static class Program
MaxSampleRate = -1, MaxSampleRate = -1,
StrictTitle = false, StrictTitle = false,
StrictArtist = false, StrictArtist = false,
DangerWords = new string[] { "remix", " edit", "cover" }, DangerWords = new string[] { "remix", " edit", "cover", "karaoke" },
BannedUsers = { }, BannedUsers = { },
AcceptNoLength = true, AcceptNoLength = true,
}; };
@ -113,6 +119,7 @@ static class Program
static int maxConcurrentProcesses = 2; static int maxConcurrentProcesses = 2;
static int maxRetriesPerTrack = 30; static int maxRetriesPerTrack = 30;
static int maxResultsPerUser = 30; static int maxResultsPerUser = 30;
static double levenshteinWeight = 0.5;
static bool slowConsoleOutput = false; static bool slowConsoleOutput = false;
static object consoleLock = new object(); static object consoleLock = new object();
@ -124,19 +131,24 @@ static class Program
static bool noModifyShareCount = false; static bool noModifyShareCount = false;
static bool printResultsFull = false; static bool printResultsFull = false;
static bool debugPrintTracksFull = false; static bool debugPrintTracksFull = false;
static bool useRandomLogin = false;
static int searchesPerTime = 34;
static int searchResetTime = 220;
static RateLimitedSemaphore? searchSemaphore;
static string inputType = ""; static string inputType = "";
static void PrintHelp() static void PrintHelp()
{ {
// undocumented options (will likely be removed): // undocumented options:
// --m3u-only, --yt-dlp-f, --slow-output, // --m3u-only, --yt-dlp-f, --slow-output,
// --no-modify-share-count, --max-retries, --max-results-per-user, --album-search // --no-modify-share-count, --max-retries, --max-results-per-user, --album-search
// --artist-col, --title-col, --album-col, --length-col, --yt-desc-col, --yt-id-col // --artist-col, --title-col, --album-col, --length-col, --yt-desc-col, --yt-id-col
// --remove-brackets, --spotify, --csv, --string, --youtube // --remove-brackets, --spotify, --csv, --string, --youtube, --random-login
Console.WriteLine("Usage: slsk-batchdl -i <input> [OPTIONS]" + Console.WriteLine("Usage: slsk-batchdl <input> [OPTIONS]" +
"\n" + "\n" +
"\n -i --input <input> <input> is one of the following:" + "\n <input> <input> is one of the following:" +
"\n" + "\n" +
"\n Spotify playlist url or \"spotify-likes\": Download a spotify" + "\n Spotify playlist url or \"spotify-likes\": Download a spotify" +
"\n playlist or your liked songs. --spotify-id and" + "\n playlist or your liked songs. --spotify-id and" +
@ -191,7 +203,7 @@ static class Program
"\n --m3u Create an m3u8 playlist file" + "\n --m3u Create an m3u8 playlist file" +
"\n" + "\n" +
"\n --format <format> Accepted file format(s), comma-separated" + "\n --format <format> Accepted file format(s), comma-separated" +
"\n --length-tol <tol> Length tolerance in seconds (default: 3)" + "\n --length-tol <sec> Length tolerance in seconds (default: 3)" +
"\n --min-bitrate <rate> Minimum file bitrate" + "\n --min-bitrate <rate> Minimum file bitrate" +
"\n --max-bitrate <rate> Maximum file bitrate" + "\n --max-bitrate <rate> Maximum file bitrate" +
"\n --max-samplerate <rate> Maximum file sample rate" + "\n --max-samplerate <rate> Maximum file sample rate" +
@ -202,7 +214,7 @@ static class Program
"\n both search result and track title or in neither of the" + "\n both search result and track title or in neither of the" +
"\n two. Case-insensitive. (default:\"remix, edit,cover\")" + "\n two. Case-insensitive. (default:\"remix, edit,cover\")" +
"\n --pref-format <format> Preferred file format(s), comma-separated (default: mp3)" + "\n --pref-format <format> Preferred file format(s), comma-separated (default: mp3)" +
"\n --pref-length-tol <tol> Preferred length tolerance in seconds (default: 2)" + "\n --pref-length-tol <sec> Preferred length tolerance in seconds (default: 2)" +
"\n --pref-min-bitrate <rate> Preferred minimum bitrate (default: 200)" + "\n --pref-min-bitrate <rate> Preferred minimum bitrate (default: 200)" +
"\n --pref-max-bitrate <rate> Preferred maximum bitrate (default: 2200)" + "\n --pref-max-bitrate <rate> Preferred maximum bitrate (default: 2200)" +
"\n --pref-max-samplerate <rate> Preferred maximum sample rate (default: 96000)" + "\n --pref-max-samplerate <rate> Preferred maximum sample rate (default: 96000)" +
@ -210,7 +222,7 @@ static class Program
"\n --pref-banned-users <list> Comma-separated list of users to deprioritize" + "\n --pref-banned-users <list> Comma-separated list of users to deprioritize" +
"\n --pref-danger-words <list> Comma-separated list of words that should appear in either" + "\n --pref-danger-words <list> Comma-separated list of words that should appear in either" +
"\n both search result and track title or in neither of the" + "\n both search result and track title or in neither of the" +
"\n two. (default: \"mix,dj , edit,cover\")" + "\n two. (default: \"mix,dj , edit,cover,(\")" +
"\n" + "\n" +
"\n -s --skip-existing Skip if a track matching file conditions is found in the" + "\n -s --skip-existing Skip if a track matching file conditions is found in the" +
"\n output folder or your music library (if provided)" + "\n output folder or your music library (if provided)" +
@ -231,13 +243,21 @@ static class Program
"\n --artist-search Also try to find track by searching for the artist only" + "\n --artist-search Also try to find track by searching for the artist only" +
"\n --no-diacr-search Also perform a search without diacritics" + "\n --no-diacr-search Also perform a search without diacritics" +
"\n --no-regex-search <regex> Also perform a search without a regex pattern" + "\n --no-regex-search <regex> Also perform a search without a regex pattern" +
"\n --levenshtein-weight <num> Results are sorted by the distance between the filename" +
"\n and track title times the weight (among other things). 1" +
"\n means each differing character will downrank the result, 0" +
"\n disables this part of the sorting algorithm. (default: 0.5)" +
"\n --yt-dlp Use yt-dlp to download tracks that weren't found on" + "\n --yt-dlp Use yt-dlp to download tracks that weren't found on" +
"\n Soulseek. yt-dlp must be available from the command line." + "\n Soulseek. yt-dlp must be available from the command line." +
"\n" + "\n" +
"\n --config <path> Manually specify config file location" + "\n --config <path> Manually specify config file location" +
"\n --search-timeout <ms> Max search time in ms (default: 6000)" + "\n --search-timeout <ms> Max search time in ms (default: 6000)" +
"\n --max-stale-time <ms> Max download time without progress in ms (default: 50000)" + "\n --max-stale-time <ms> Max download time without progress in ms (default: 50000)" +
"\n --concurrent-downloads <num> Max concurrent searches & downloads (default: 2)" + "\n --concurrent-downloads <num> Max concurrent downloads (default: 2)" +
"\n --searches-per-time <num> Max searches per time interval. Higher values may cause" +
"\n 30-minute bans. (default: 34)" +
"\n --searches-time <sec> Controls how often available searches are replenished." +
"\n Lower values may cause 30-minute bans. (default: 220)" +
"\n --display <option> Changes how searches and downloads are displayed:" + "\n --display <option> Changes how searches and downloads are displayed:" +
"\n single (default): Show transfer state and percentage" + "\n single (default): Show transfer state and percentage" +
"\n double: Transfer state and a large progress bar " + "\n double: Transfer state and a large progress bar " +
@ -287,6 +307,8 @@ static class Program
} }
for (int i = 0; i < args.Length; i++) for (int i = 0; i < args.Length; i++)
{
if (args[i].StartsWith("-"))
{ {
switch (args[i]) switch (args[i])
{ {
@ -308,7 +330,6 @@ static class Program
break; break;
case "-p": case "-p":
case "--path": case "--path":
case "--parent":
parentFolder = args[++i]; parentFolder = args[++i];
break; break;
case "--config": case "--config":
@ -348,6 +369,9 @@ static class Program
case "--password": case "--password":
password = args[++i]; password = args[++i];
break; break;
case "--random-login":
useRandomLogin = true;
break;
case "--artist-col": case "--artist-col":
artistCol = args[++i]; artistCol = args[++i];
break; break;
@ -387,7 +411,8 @@ static class Program
debugPrintTracks = true; debugPrintTracks = true;
debugDisableDownload = true; debugDisableDownload = true;
} }
else if (opt == "tracks-full") { else if (opt == "tracks-full")
{
debugPrintTracks = true; debugPrintTracks = true;
debugPrintTracksFull = true; debugPrintTracksFull = true;
debugDisableDownload = true; debugDisableDownload = true;
@ -460,6 +485,12 @@ static class Program
case "--concurrent-downloads": case "--concurrent-downloads":
maxConcurrentProcesses = int.Parse(args[++i]); maxConcurrentProcesses = int.Parse(args[++i]);
break; break;
case "--searches-per-time":
searchesPerTime = int.Parse(args[++i]);
break;
case "--searches-time":
searchResetTime = int.Parse(args[++i]);
break;
case "--max-retries": case "--max-retries":
maxRetriesPerTrack = int.Parse(args[++i]); maxRetriesPerTrack = int.Parse(args[++i]);
break; break;
@ -520,6 +551,9 @@ static class Program
case "--banned-users": case "--banned-users":
necessaryCond.BannedUsers = args[++i].Split(','); necessaryCond.BannedUsers = args[++i].Split(',');
break; break;
case "--levenshtein-weight":
levenshteinWeight = double.Parse(args[++i]);
break;
case "--slow-output": case "--slow-output":
slowConsoleOutput = true; slowConsoleOutput = true;
break; break;
@ -570,9 +604,17 @@ static class Program
throw new ArgumentException($"Unknown argument: {args[i]}"); throw new ArgumentException($"Unknown argument: {args[i]}");
} }
} }
else
{
if (input == "")
input = args[i];
else
throw new ArgumentException();
}
}
if (input == "") if (input == "")
throw new ArgumentException($"Must provide an -i argument."); throw new ArgumentException($"No input provided");
if (ytKey != "") if (ytKey != "")
YouTube.apiKey = ytKey; YouTube.apiKey = ytKey;
@ -580,6 +622,8 @@ static class Program
if (debugDisableDownload) if (debugDisableDownload)
maxConcurrentProcesses = 1; maxConcurrentProcesses = 1;
searchSemaphore = new RateLimitedSemaphore(searchesPerTime, TimeSpan.FromSeconds(searchResetTime));
int max = reverse ? int.MaxValue : maxTracks; int max = reverse ? int.MaxValue : maxTracks;
int off = reverse ? 0 : offset; int off = reverse ? 0 : offset;
@ -712,10 +756,8 @@ static class Program
if (folderName == "") if (folderName == "")
folderName = ReplaceInvalidChars(searchStr, " "); folderName = ReplaceInvalidChars(searchStr, " ");
var music = ParseTrackArg(searchStr); var music = ParseTrackArg(searchStr);
await WaitForInternetConnection();
await client.ConnectAsync(username, password); await Login();
if (!noModifyShareCount)
await client.SetSharedCountsAsync(10, 50);
var x = new List<string>(); var x = new List<string>();
if (music.ArtistName != "") if (music.ArtistName != "")
@ -847,15 +889,11 @@ static class Program
Console.WriteLine($"Downloading {tracks.Count} tracks{skippedTracks}\n"); Console.WriteLine($"Downloading {tracks.Count} tracks{skippedTracks}\n");
} }
if (string.IsNullOrEmpty(username) || string.IsNullOrEmpty(password)) if (!useRandomLogin && (string.IsNullOrEmpty(username) || string.IsNullOrEmpty(password)))
throw new Exception("No soulseek username or password"); throw new ArgumentException("No soulseek username or password");
await WaitForInternetConnection(); if (!client.State.HasFlag(SoulseekClientStates.LoggedIn))
if (!aggregate) { await Login(useRandomLogin);
await client.ConnectAsync(username, password);
if (!noModifyShareCount)
await client.SetSharedCountsAsync(10, 50);
}
int successCount = 0; int successCount = 0;
int failCount = 0; int failCount = 0;
@ -870,7 +908,7 @@ static class Program
retry: retry:
try try
{ {
await WaitForInternetConnection(); await WaitForNetworkAndLogin();
var savedFilePath = await SearchAndDownload(track); var savedFilePath = await SearchAndDownload(track);
Interlocked.Increment(ref successCount); Interlocked.Increment(ref successCount);
if (removeTracksFromSource && !string.IsNullOrEmpty(spotifyUrl)) if (removeTracksFromSource && !string.IsNullOrEmpty(spotifyUrl))
@ -878,24 +916,29 @@ static class Program
if (createM3u && !debugDisableDownload) if (createM3u && !debugDisableDownload)
m3uEditor.WriteSuccess(savedFilePath, track); m3uEditor.WriteSuccess(savedFilePath, track);
} }
catch (SearchAndDownloadException ex) catch (Exception ex)
{
if (ex is SearchAndDownloadException)
{ {
Interlocked.Increment(ref failCount); Interlocked.Increment(ref failCount);
if (!debugDisableDownload && inputType != "string") if (!debugDisableDownload && inputType != "string")
m3uEditor.WriteFail(ex.Message, track); m3uEditor.WriteFail(ex.Message, track);
} }
catch (Exception ex) else if (!client.State.HasFlag(SoulseekClientStates.LoggedIn))
{ {
goto retry;
}
else
{
WriteLine($"\n{ex.Message}\n{ex.StackTrace}\n", ConsoleColor.DarkYellow, true);
if (tries-- > 0) if (tries-- > 0)
goto retry; goto retry;
Interlocked.Increment(ref failCount); Interlocked.Increment(ref failCount);
WriteLine($"\n{ex.Message}\n", ConsoleColor.DarkYellow, true); }
} }
finally { semaphore.Release(); } finally { semaphore.Release(); }
if ((DateTime.Now - lastUpdate).TotalMilliseconds > updateDelay * 3) if ((successCount + failCount + 1) % 50 == 0)
UpdateTask = Task.Run(() => Update());
else if ((successCount + failCount + 1) % 50 == 0)
WriteLine($"\nSuccesses: {successCount}, fails: {failCount}, tracks left: {tracksRemaining}\n", ConsoleColor.Yellow, true); WriteLine($"\nSuccesses: {successCount}, fails: {failCount}, tracks left: {tracksRemaining}\n", ConsoleColor.Yellow, true);
Interlocked.Decrement(ref tracksRemaining); Interlocked.Decrement(ref tracksRemaining);
@ -910,6 +953,35 @@ static class Program
} }
static async Task Login(bool random=false, int tries=3)
{
string user = username, pass = password;
if (random)
{
var r = new Random();
const string chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
user = new string(Enumerable.Repeat(chars, 10).Select(s => s[r.Next(s.Length)]).ToArray());
pass = new string(Enumerable.Repeat(chars, 10).Select(s => s[r.Next(s.Length)]).ToArray());
}
while (true)
{
try
{
await WaitForInternetConnection();
await client.ConnectAsync(user, pass);
if (!noModifyShareCount)
await client.SetSharedCountsAsync(10, 50);
break;
}
catch {
if (--tries == 0)
throw;
}
}
}
static async Task<string> SearchAndDownload(Track track) static async Task<string> SearchAndDownload(Track track)
{ {
Console.ResetColor(); Console.ResetColor();
@ -928,7 +1000,7 @@ static class Program
goto downloads; goto downloads;
} }
RefreshOrPrint(progress, 0, $"Searching: {track}", true); RefreshOrPrint(progress, 0, $"Waiting: {track}", false);
var title = track.ArtistName != "" ? $"{track.ArtistName} - {track.TrackTitle}" : $"{track.TrackTitle}"; var title = track.ArtistName != "" ? $"{track.ArtistName} - {track.TrackTitle}" : $"{track.TrackTitle}";
string searchText = $"{title}"; string searchText = $"{title}";
@ -1000,7 +1072,8 @@ static class Program
var responseHandlerUncapped = getResponseHandler(preferredCond); var responseHandlerUncapped = getResponseHandler(preferredCond);
var searchOptions = getSearchOptions(searchTimeout, necessaryCond); var searchOptions = getSearchOptions(searchTimeout, necessaryCond);
await RunSearches(searchText, searchOptions, responseHandler, cts.Token); var onSearch = () => RefreshOrPrint(progress, 0, $"Searching: {track}", true);
await RunSearches(searchText, searchOptions, responseHandler, cts.Token, onSearch);
if (results.Count == 0 && albumSearchTrack && track.Album != "") if (results.Count == 0 && albumSearchTrack && track.Album != "")
{ {
@ -1009,8 +1082,9 @@ static class Program
necCond2.StrictTitle = true; necCond2.StrictTitle = true;
searchOptions = getSearchOptions(Math.Min(5000, searchTimeout), necCond2); searchOptions = getSearchOptions(Math.Min(5000, searchTimeout), necCond2);
RefreshOrPrint(progress, 0, $"Searching (album name): {track.Album}"); RefreshOrPrint(progress, 0, $"Waiting (album name search): {track}");
await RunSearches(searchText, searchOptions, responseHandlerUncapped, cts.Token); onSearch = () => RefreshOrPrint(progress, 0, $"Searching (album name): {track}");
await RunSearches(searchText, searchOptions, responseHandlerUncapped, cts.Token, onSearch);
} }
if (results.Count == 0 && (noArtistSearchTrack || track.ArtistMaybeWrong) && !string.IsNullOrEmpty(track.ArtistName)) if (results.Count == 0 && (noArtistSearchTrack || track.ArtistMaybeWrong) && !string.IsNullOrEmpty(track.ArtistName))
@ -1021,8 +1095,9 @@ static class Program
necCond2.StrictArtist = true; necCond2.StrictArtist = true;
searchOptions = getSearchOptions(Math.Min(5000, searchTimeout), necCond2); searchOptions = getSearchOptions(Math.Min(5000, searchTimeout), necCond2);
RefreshOrPrint(progress, 0, $"Searching (no artist name): {searchText}"); RefreshOrPrint(progress, 0, $"Waiting (no artist name search): {track}");
await RunSearches(searchText, searchOptions, responseHandlerUncapped, cts.Token); onSearch = () => RefreshOrPrint(progress, 0, $"Searching (no artist name): {track}");
await RunSearches(searchText, searchOptions, responseHandlerUncapped, cts.Token, onSearch);
} }
if (results.Count == 0 && artistSearchTrack && !string.IsNullOrEmpty(track.ArtistName)) if (results.Count == 0 && artistSearchTrack && !string.IsNullOrEmpty(track.ArtistName))
@ -1032,8 +1107,9 @@ static class Program
necCond2.StrictTitle = true; necCond2.StrictTitle = true;
searchOptions = getSearchOptions(Math.Min(6000, searchTimeout), necCond2); searchOptions = getSearchOptions(Math.Min(6000, searchTimeout), necCond2);
RefreshOrPrint(progress, 0, $"Searching (artist name): {searchText}"); RefreshOrPrint(progress, 0, $"Waiting (artist name search): {track}");
await RunSearches(searchText, searchOptions, responseHandlerUncapped, cts.Token); onSearch = () => RefreshOrPrint(progress, 0, $"Searching (artist name): {track}");
await RunSearches(searchText, searchOptions, responseHandlerUncapped, cts.Token, onSearch);
} }
lock (downloadingLocker) { } lock (downloadingLocker) { }
@ -1063,7 +1139,7 @@ static class Program
else if (!downloading && results.Count > 0) else if (!downloading && results.Count > 0)
{ {
var random = new Random(); var random = new Random();
var fileResponses = OrderedResults(results, track, badUsers); var fileResponses = OrderedResults(results, track, badUsers, true);
if (debugDisableDownload) if (debugDisableDownload)
{ {
@ -1075,8 +1151,15 @@ static class Program
return ""; return "";
} }
var newBadUsers = new ConcurrentBag<string>();
var ignoredResults = new ConcurrentDictionary<string, (SearchResponse, Soulseek.File)>();
foreach (var x in fileResponses) foreach (var x in fileResponses)
{ {
if (newBadUsers.Contains(x.response.Username))
{
ignoredResults.TryAdd(x.response.Username + "\\" + x.file.Filename, (x.response, x.file));
continue;
}
saveFilePath = GetSavePath(x.file.Filename, track); saveFilePath = GetSavePath(x.file.Filename, track);
try try
{ {
@ -1087,6 +1170,9 @@ static class Program
catch catch
{ {
downloading = false; downloading = false;
if (!client.State.HasFlag(SoulseekClientStates.LoggedIn))
throw;
newBadUsers.Add(x.response.Username);
if (--maxRetriesPerTrack <= 0) if (--maxRetriesPerTrack <= 0)
{ {
RefreshOrPrint(progress, 0, $"Out of download retries: {track}, skipping", true); RefreshOrPrint(progress, 0, $"Out of download retries: {track}, skipping", true);
@ -1193,28 +1279,49 @@ static class Program
string trackName = track.TrackTitle.Trim(); string trackName = track.TrackTitle.Trim();
string albumName = track.Album.Trim(); string albumName = track.Album.Trim();
var fileResponses = results.Select(x => x.Value);
var equivalentFiles = EquivalentFiles(track, fileResponses);
var tracks = equivalentFiles
.Select(kvp => {
kvp.Item1.Downloads = new ConcurrentDictionary<string, (SearchResponse, Soulseek.File)>(
kvp.Item2.ToDictionary(item => { return item.response.Username + "\\" + item.file.Filename; }, item => item));
return kvp.Item1;
}).ToList();
return tracks;
}
static IOrderedEnumerable<(Track, IEnumerable<(SearchResponse response, Soulseek.File file)>)> EquivalentFiles(Track track, IEnumerable<(SearchResponse, Soulseek.File)> fileResponses, int minShares=-1)
{
if (minShares == -1)
minShares = minUsersAggregate;
var inferTrack = ((SearchResponse r, Soulseek.File f) x) => { var inferTrack = ((SearchResponse r, Soulseek.File f) x) => {
Track t = track; Track t = track;
t.Length = x.f.Length ?? -1; t.Length = x.f.Length ?? -1;
return InferTrack(x.f.Filename, t); return InferTrack(x.f.Filename, t);
}; };
var fileResponses = OrderedResults(results, track, new string[0]); var res = fileResponses
var equivalentFiles = fileResponses
.GroupBy(inferTrack, new TrackStringComparer(ignoreCase: true)) .GroupBy(inferTrack, new TrackStringComparer(ignoreCase: true))
.Where(group => group.Select(x => x.Item1.Username).Distinct().Count() >= minUsersAggregate) .Where(group => group.Select(x => x.Item1.Username).Distinct().Count() >= minShares)
.SelectMany(group => { .SelectMany(group => {
var sortedTracks = group.OrderBy(t => t.Item2.Length).Where(x => x.Item2.Length != null).ToList(); var sortedTracks = group.OrderBy(t => t.Item2.Length).Where(x => x.Item2.Length != null).ToList();
var groups = new List<(Track, List<(SearchResponse, Soulseek.File)>)>(); var groups = new List<(Track, List<(SearchResponse, Soulseek.File)>)>();
var noLengthGroup = group.Where(x => x.Item2.Length == null); var noLengthGroup = group.Where(x => x.Item2.Length == null);
for (int i = 0; i < sortedTracks.Count;) { for (int i = 0; i < sortedTracks.Count;)
{
var subGroup = new List<(SearchResponse, Soulseek.File)> { sortedTracks[i] }; var subGroup = new List<(SearchResponse, Soulseek.File)> { sortedTracks[i] };
int j = i + 1; int j = i + 1;
while (j < sortedTracks.Count) { while (j < sortedTracks.Count)
{
int l1 = (int)sortedTracks[j].Item2.Length; int l1 = (int)sortedTracks[j].Item2.Length;
int l2 = (int)sortedTracks[i].Item2.Length; int l2 = (int)sortedTracks[i].Item2.Length;
if (Math.Abs(l1 - l2) <= necessaryCond.LengthTolerance) { if (Math.Abs(l1 - l2) <= necessaryCond.LengthTolerance)
{
subGroup.Add(sortedTracks[j]); subGroup.Add(sortedTracks[j]);
j++; j++;
} }
@ -1226,37 +1333,60 @@ static class Program
i = j; i = j;
} }
if (noLengthGroup.Count() > 0) { if (noLengthGroup.Count() > 0)
{
if (groups.Count() > 0 && !preferredCond.AcceptNoLength) if (groups.Count() > 0 && !preferredCond.AcceptNoLength)
groups.First().Item2.AddRange(noLengthGroup); groups.First().Item2.AddRange(noLengthGroup);
else else
groups.Add((group.Key, noLengthGroup.ToList())); groups.Add((group.Key, noLengthGroup.ToList()));
} }
return groups.Where(subGroup => subGroup.Item2.Select(x => x.Item1.Username).Distinct().Count() >= minUsersAggregate) return groups.Where(subGroup => subGroup.Item2.Select(x => x.Item1.Username).Distinct().Count() >= minShares)
.Select(subGroup => (subGroup.Item1, OrderedResults(subGroup.Item2 .Select(subGroup => (subGroup.Item1, subGroup.Item2.AsEnumerable()));
.Select(item => new KeyValuePair<string, (SearchResponse, Soulseek.File)>(subGroup.Item1.ToString(), item)), subGroup.Item1, new string[0]))); }).OrderByDescending(x => x.Item2.Count());
});
return res;
var tracks = equivalentFiles.Select(kvp => {
kvp.Item1.Downloads = new ConcurrentDictionary<string, (SearchResponse, Soulseek.File)>(
kvp.Item2.ToDictionary(item => { return item.response.Username + "\\" + item.file.Filename; }, item => item));
return kvp.Item1; })
.ToList();
return tracks;
} }
static IOrderedEnumerable<(SearchResponse response, Soulseek.File file)> OrderedResults(IEnumerable<KeyValuePair<string, (SearchResponse, Soulseek.File)>> results, Track track, IEnumerable<string> ignoreUsers) static IOrderedEnumerable<(SearchResponse response, Soulseek.File file)> OrderedResults(IEnumerable<KeyValuePair<string, (SearchResponse, Soulseek.File)>> results, Track track, IEnumerable<string> ignoreUsers, bool useInfer=false)
{ {
Dictionary<string, (Track, int)>? result = null;
if (useInfer)
{
var equivalentFiles = EquivalentFiles(track, results.Select(x => x.Value), 1);
result = equivalentFiles
.SelectMany(t => t.Item2, (t, f) => new { t.Item1, f.response.Username, f.file.Filename })
.GroupBy(x => $"{x.Username}\\{x.Filename}")
.ToSafeDictionary(
g => g.Key,
g => (g.First().Item1, g.Count()));
}
var infTrack = (string fname, string uname) => {
string key = $"{uname}\\{fname}";
if (result != null && result.ContainsKey(key))
return result[key];
return (new Track(), 0);
};
string t1 = track.TrackTitle.ReplaceInvalidChars("").Replace("—", "-").Replace("_", "").RemoveFt().RemoveDiacritics().Replace(" ", "").ToLower();
var value = (Track inferredTrack) => {
if (levenshteinWeight == 0)
return 100;
string t2 = inferredTrack.TrackTitle.ReplaceInvalidChars("").Replace("—", "-").Replace("_", "").RemoveFt().RemoveDiacritics().Replace(" ", "").ToLower();
int val = t2.Contains(t1) || track.ArtistMaybeWrong ? 100 : 50;
return (int)(val - Utils.Levenshtein(t1, t2) * levenshteinWeight);
};
var random = new Random(); var random = new Random();
return results return results.Select(kvp => (response: kvp.Value.Item1, file: kvp.Value.Item2))
.Select(kvp => (response: kvp.Value.Item1, file: kvp.Value.Item2))
.OrderByDescending(x => !ignoreUsers.Contains(x.response.Username)) .OrderByDescending(x => !ignoreUsers.Contains(x.response.Username))
.ThenByDescending(x => x.file.Length != null || preferredCond.AcceptNoLength) .ThenByDescending(x => (x.file.Length != null && x.file.Length > 0) || preferredCond.AcceptNoLength)
.ThenByDescending(x => preferredCond.BannedUsersSatisfies(x.response)) .ThenByDescending(x => preferredCond.BannedUsersSatisfies(x.response))
.ThenByDescending(x => value(infTrack(x.file.Filename, x.response.Username).Item1))
.ThenByDescending(x => preferredCond.DangerWordSatisfies(x.file.Filename, track.TrackTitle, track.ArtistName))
.ThenByDescending(x => preferredCond.StrictTitleSatisfies(x.file.Filename, track.TrackTitle)) .ThenByDescending(x => preferredCond.StrictTitleSatisfies(x.file.Filename, track.TrackTitle))
.ThenByDescending(x => preferredCond.LengthToleranceSatisfies(x.file, track.Length)) .ThenByDescending(x => preferredCond.LengthToleranceSatisfies(x.file, track.Length))
.ThenByDescending(x => preferredCond.BitrateSatisfies(x.file)) .ThenByDescending(x => preferredCond.BitrateSatisfies(x.file))
@ -1264,33 +1394,37 @@ static class Program
.ThenByDescending(x => preferredCond.FileSatisfies(x.file, track, x.response)) .ThenByDescending(x => preferredCond.FileSatisfies(x.file, track, x.response))
.ThenByDescending(x => x.response.HasFreeUploadSlot) .ThenByDescending(x => x.response.HasFreeUploadSlot)
.ThenByDescending(x => necessaryCond.FileSatisfies(x.file, track, x.response)) .ThenByDescending(x => necessaryCond.FileSatisfies(x.file, track, x.response))
.ThenByDescending(x => FileConditions.StrictString(x.file.Filename, track.TrackTitle))
.ThenByDescending(x => x.response.UploadSpeed / 600) .ThenByDescending(x => x.response.UploadSpeed / 600)
.ThenByDescending(x => FileConditions.StrictString(x.file.Filename, track.TrackTitle))
.ThenByDescending(x => FileConditions.StrictString(x.file.Filename, track.ArtistName)) .ThenByDescending(x => FileConditions.StrictString(x.file.Filename, track.ArtistName))
.ThenByDescending(x => track.Length > 0 ? -Math.Max(Math.Abs(track.Length - (x.file.Length ?? -9999)) - 1, 0) : 0) .ThenByDescending(x => track.Length > 0 ? -Math.Max(Math.Abs(track.Length - (x.file.Length ?? -9999)) - 1, 0) / 3 : 0)
.ThenByDescending(x => FileConditions.StrictString(x.file.Filename, track.Album))
.ThenByDescending(x => x.response.UploadSpeed / 300) .ThenByDescending(x => x.response.UploadSpeed / 300)
.ThenByDescending(x => (x.file.BitRate ?? 0) / 70) .ThenByDescending(x => (x.file.BitRate ?? 0) / 70)
.ThenByDescending(x => infTrack(x.file.Filename, x.response.Username).Item2)
.ThenByDescending(x => random.Next()); .ThenByDescending(x => random.Next());
} }
static async Task RunSearches(string search, SearchOptions opts, Action<SearchResponse> rHandler, CancellationToken ct) static async Task RunSearches(string search, SearchOptions opts, Action<SearchResponse> rHandler, CancellationToken ct, Action? onSearch=null)
{ {
await searchSemaphore.WaitAsync();
await WaitForInternetConnection();
try try
{ {
var q = SearchQuery.FromText(search); var q = SearchQuery.FromText(search);
await WaitForInternetConnection();
var searchTasks = new List<Task>(); var searchTasks = new List<Task>();
onSearch?.Invoke();
searchTasks.Add(client.SearchAsync(q, options: opts, cancellationToken: ct, responseHandler: rHandler)); searchTasks.Add(client.SearchAsync(q, options: opts, cancellationToken: ct, responseHandler: rHandler));
if (noDiacrSearch && search.RemoveDiacriticsIfExist(out string newSearch)) if (noDiacrSearch && search.RemoveDiacriticsIfExist(out string newSearch))
{ {
await searchSemaphore.WaitAsync();
var searchQuery2 = SearchQuery.FromText(newSearch); var searchQuery2 = SearchQuery.FromText(newSearch);
searchTasks.Add(client.SearchAsync(searchQuery2, options: opts, cancellationToken: ct, responseHandler: rHandler)); searchTasks.Add(client.SearchAsync(searchQuery2, options: opts, cancellationToken: ct, responseHandler: rHandler));
} }
if (!string.IsNullOrEmpty(noRegexSearch) && search.RemoveRegexIfExist(noRegexSearch, out string newSearch2)) if (!string.IsNullOrEmpty(noRegexSearch) && search.RemoveRegexIfExist(noRegexSearch, out string newSearch2))
{ {
await searchSemaphore.WaitAsync();
var searchQuery2 = SearchQuery.FromText(newSearch2); var searchQuery2 = SearchQuery.FromText(newSearch2);
searchTasks.Add(client.SearchAsync(searchQuery2, options: opts, cancellationToken: ct, responseHandler: rHandler)); searchTasks.Add(client.SearchAsync(searchQuery2, options: opts, cancellationToken: ct, responseHandler: rHandler));
} }
@ -1481,20 +1615,20 @@ static class Program
downloads.TryRemove(file.Filename, out _); downloads.TryRemove(file.Filename, out _);
} }
static int totalCalls = 0;
static async Task Update() static async Task Update()
{ {
totalCalls++;
int thisCall = totalCalls;
if (slowConsoleOutput) if (slowConsoleOutput)
updateDelay = slowUpdateDelay; updateDelay = slowUpdateDelay;
while (thisCall == totalCalls) while (true)
{ {
lastUpdate = DateTime.Now; lastUpdate = DateTime.Now;
if (!skipUpdate) if (!skipUpdate)
{
try
{
if (client.State.HasFlag(SoulseekClientStates.LoggedIn))
{ {
foreach (var (key, val) in searches) foreach (var (key, val) in searches)
if (val == null) searches.TryRemove(key, out _); if (val == null) searches.TryRemove(key, out _);
@ -1518,6 +1652,22 @@ static class Program
else downloads.TryRemove(key, out _); else downloads.TryRemove(key, out _);
} }
} }
else if (!client.State.HasFlag(SoulseekClientStates.LoggedIn | SoulseekClientStates.LoggingIn | SoulseekClientStates.Connecting))
{
WriteLine($"\nDisconnected, logging in\n", ConsoleColor.DarkYellow, true);
try { await Login(useRandomLogin); }
catch (Exception ex)
{
string banMsg = useRandomLogin ? "" : " (likely a 30-minute ban caused by frequent searches)";
WriteLine($"{ex.Message}{banMsg}", ConsoleColor.DarkYellow, true);
}
}
}
catch (Exception ex)
{
WriteLine($"\n{ex.Message}\n", ConsoleColor.DarkYellow, true);
}
}
await Task.Delay(updateDelay); await Task.Delay(updateDelay);
} }
@ -1683,7 +1833,7 @@ static class Program
updatedTextSuccess |= success; updatedTextSuccess |= success;
Console.ResetColor(); Console.ResetColor();
RefreshOrPrint(progress, (int)((percentage ?? 0) * 100), $"{txt} {displayText}", needSimplePrintUpdate); RefreshOrPrint(progress, (int)((percentage ?? 0) * 100), $"{txt} {displayText}", needSimplePrintUpdate, needSimplePrintUpdate);
} }
@ -1731,7 +1881,8 @@ static class Program
public FileConditions(FileConditions other) public FileConditions(FileConditions other)
{ {
other.Formats.CopyTo(Formats, 0); Array.Resize(ref Formats, other.Formats.Length);
Array.Copy(other.Formats, Formats, other.Formats.Length);
LengthTolerance = other.LengthTolerance; LengthTolerance = other.LengthTolerance;
MinBitrate = other.MinBitrate; MinBitrate = other.MinBitrate;
MaxBitrate = other.MaxBitrate; MaxBitrate = other.MaxBitrate;
@ -1814,7 +1965,7 @@ static class Program
if (string.IsNullOrEmpty(fname) || string.IsNullOrEmpty(tname)) if (string.IsNullOrEmpty(fname) || string.IsNullOrEmpty(tname))
return false; return false;
string pattern = $@"(?i:(?<=[. -\/\\]|^){tname}(?=[. -\/\\]|$))"; string pattern = $@"(?i:(?<=[. -\/\\]|^){Regex.Escape(tname)}(?=[. -\/\\]|$))";
return Regex.IsMatch(fname, pattern); return Regex.IsMatch(fname, pattern);
} }
@ -2399,7 +2550,8 @@ static class Program
string bitRate = file.BitRate.HasValue ? $"/{file.BitRate}kbps" : ""; string bitRate = file.BitRate.HasValue ? $"/{file.BitRate}kbps" : "";
string fileSize = $"{file.Size / (float)(1024 * 1024):F1}MB"; string fileSize = $"{file.Size / (float)(1024 * 1024):F1}MB";
string fname = fullpath ? "\\" + file.Filename : "\\..\\" + GetFileNameSlsk(file.Filename); string fname = fullpath ? "\\" + file.Filename : "\\..\\" + GetFileNameSlsk(file.Filename);
string displayText = $"{response?.Username ?? ""}{fname} [{file.Length}s{sampleRate}{bitRate}/{fileSize}]"; string length = (file.Length ?? -1).ToString();
string displayText = $"{response?.Username ?? ""}{fname} [{length}s{sampleRate}{bitRate}/{fileSize}]";
string necStr = nec != null ? $"nec:{nec.GetNotSatisfiedName(file, t, response)}, " : ""; string necStr = nec != null ? $"nec:{nec.GetNotSatisfiedName(file, t, response)}, " : "";
string prefStr = pref != null ? $"prf:{pref.GetNotSatisfiedName(file, t, response)}" : ""; string prefStr = pref != null ? $"prf:{pref.GetNotSatisfiedName(file, t, response)}" : "";
@ -2444,14 +2596,14 @@ static class Program
Console.WriteLine($" ... (etc)"); Console.WriteLine($" ... (etc)");
} }
static void RefreshOrPrint(ProgressBar? progress, int current, string item, bool print = false) static void RefreshOrPrint(ProgressBar? progress, int current, string item, bool print = false, bool refreshIfOffscreen = false)
{ {
if (progress != null) if (progress != null && !Console.IsOutputRedirected && (refreshIfOffscreen || progress.Y >= Console.WindowTop))
{ {
try { progress.Refresh(current, item); } try { progress.Refresh(current, item); }
catch { } catch { }
} }
else if (displayStyle == "simple" && print) else if ((displayStyle == "simple" || Console.IsOutputRedirected) && print)
Console.WriteLine(item); Console.WriteLine(item);
} }
@ -2516,6 +2668,18 @@ static class Program
await Task.Delay(500); await Task.Delay(500);
} }
} }
public static async Task WaitForNetworkAndLogin()
{
await WaitForInternetConnection();
while (true)
{
if (client.State.HasFlag(SoulseekClientStates.LoggedIn))
break;
await Task.Delay(500);
}
}
} }
public struct Track public struct Track
@ -2589,6 +2753,7 @@ class TrackStringComparer : IEqualityComparer<Track>
} }
} }
public class M3UEditor public class M3UEditor
{ {
public readonly List<Track> tracks; public readonly List<Track> tracks;
@ -2665,7 +2830,60 @@ public class M3UEditor
} }
} }
public static class ExtensionMethods
class RateLimitedSemaphore
{
private readonly int maxCount;
private readonly TimeSpan resetTimeSpan;
private readonly SemaphoreSlim semaphore;
private long nextResetTimeTicks;
private readonly object resetTimeLock = new object();
public RateLimitedSemaphore(int maxCount, TimeSpan resetTimeSpan)
{
this.maxCount = maxCount;
this.resetTimeSpan = resetTimeSpan;
this.semaphore = new SemaphoreSlim(maxCount, maxCount);
this.nextResetTimeTicks = (DateTimeOffset.UtcNow + this.resetTimeSpan).UtcTicks;
}
private void TryResetSemaphore()
{
if (!(DateTimeOffset.UtcNow.UtcTicks > Interlocked.Read(ref this.nextResetTimeTicks)))
return;
lock (this.resetTimeLock)
{
var currentTime = DateTimeOffset.UtcNow;
if (currentTime.UtcTicks > Interlocked.Read(ref this.nextResetTimeTicks))
{
this.semaphore.Release(this.maxCount - this.semaphore.CurrentCount);
var newResetTimeTicks = (currentTime + this.resetTimeSpan).UtcTicks;
Interlocked.Exchange(ref this.nextResetTimeTicks, newResetTimeTicks);
}
}
}
public async Task WaitAsync()
{
TryResetSemaphore();
var semaphoreTask = this.semaphore.WaitAsync();
while (!semaphoreTask.IsCompleted)
{
var ticks = Interlocked.Read(ref this.nextResetTimeTicks);
var nextResetTime = new DateTimeOffset(new DateTime(ticks, DateTimeKind.Utc));
var delayTime = nextResetTime - DateTimeOffset.UtcNow;
var delayTask = delayTime >= TimeSpan.Zero ? Task.Delay(delayTime) : Task.CompletedTask;
await Task.WhenAny(semaphoreTask, delayTask);
TryResetSemaphore();
}
}
}
public static class Utils
{ {
public static bool EqualsAny(this string input, string[] values, StringComparison comparison = StringComparison.Ordinal) public static bool EqualsAny(this string input, string[] values, StringComparison comparison = StringComparison.Ordinal)
{ {
@ -2754,6 +2972,46 @@ public static class ExtensionMethods
return c; return c;
} }
public static Dictionary<K, V> ToSafeDictionary<T, K, V>(this IEnumerable<T> source, Func<T, K> keySelector, Func<T, V> valSelector)
{
var d = new Dictionary<K, V>();
foreach (var element in source)
{
if (!d.ContainsKey(keySelector(element)))
d.Add(keySelector(element), valSelector(element));
}
return d;
}
public static int Levenshtein(string source, string target)
{
if (source.Length == 0)
return target.Length;
if (target.Length == 0)
return source.Length;
var distance = new int[source.Length + 1, target.Length + 1];
for (var i = 0; i <= source.Length; i++)
distance[i, 0] = i;
for (var j = 0; j <= target.Length; j++)
distance[0, j] = j;
for (var i = 1; i <= source.Length; i++)
{
for (var j = 1; j <= target.Length; j++)
{
var cost = (target[j - 1] == source[i - 1]) ? 0 : 1;
distance[i, j] = Math.Min(
Math.Min(distance[i - 1, j] + 1, distance[i, j - 1] + 1),
distance[i - 1, j - 1] + cost);
}
}
return distance[source.Length, target.Length];
}
public static string RemoveDiacritics(this string s) public static string RemoveDiacritics(this string s)
{ {
string text = ""; string text = "";

2
slsk-batchdl/slsk-batchdl.csproj
View file

@ -25,7 +25,7 @@
<PackageReference Include="SpotifyAPI.Web" Version="7.0.2" /> <PackageReference Include="SpotifyAPI.Web" Version="7.0.2" />
<PackageReference Include="SpotifyAPI.Web.Auth" Version="7.0.2" /> <PackageReference Include="SpotifyAPI.Web.Auth" Version="7.0.2" />
<PackageReference Include="TagLibSharp" Version="2.3.0" /> <PackageReference Include="TagLibSharp" Version="2.3.0" />
<PackageReference Include="YoutubeExplode" Version="6.3.7" /> <PackageReference Include="YoutubeExplode" Version="6.3.8" />
</ItemGroup> </ItemGroup>
</Project> </Project>