mirror/slsk-batchdl
1
0
Fork 0
mirror of https://github.com/fiso64/slsk-batchdl.git synced 2024-12-22 22:42:41 +00:00
Code Issues Activity

commit

Browse source
This commit is contained in:
fiso64 2024-09-01 19:52:50 +02:00
parent d56d68356a
commit 7cd634e54f
20 changed files with 849 additions and 390 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

55
README.md
View file

@ -1,6 +1,6 @@
# slsk-batchdl # slsk-batchdl
A batch downloader for Soulseek built with Soulseek.NET. Accepts CSV files as well as Spotify and YouTube urls. An automatic downloader for Soulseek built with Soulseek.NET. Accepts CSV files as well as Spotify and YouTube urls.
See the [examples](#examples-1). See the [examples](#examples-1).
## Index ## Index
@ -41,7 +41,6 @@ Usage: sldl <input> [OPTIONS]
``` ```
General Options General Options
-p, --path <path> Download directory -p, --path <path> Download directory
-f, --folder <name> Subfolder name. Set to '.' to output directly to --path
--input-type <type> Force set input type, [csv|youtube|spotify|bandcamp|string] --input-type <type> Force set input type, [csv|youtube|spotify|bandcamp|string]
--name-format <format> Name format for downloaded tracks. See --help name-format --name-format <format> Name format for downloaded tracks. See --help name-format
@ -189,9 +188,6 @@ Usage: sldl <input> [OPTIONS]
-t, --interactive Interactive mode, allows to select the folder and images -t, --interactive Interactive mode, allows to select the folder and images
--album-track-count <num> Specify the exact number of tracks in the album. Add a + or --album-track-count <num> Specify the exact number of tracks in the album. Add a + or
- for inequalities, e.g '5+' for five or more tracks. - for inequalities, e.g '5+' for five or more tracks.
--album-ignore-fails Do not skip to the next source and do not delete all
successfully downloaded files if one of the files in the
folder fails to download
--album-art <option> Retrieve additional images after downloading the album: --album-art <option> Retrieve additional images after downloading the album:
'default': No additional images 'default': No additional images
'largest': Download from the folder with the largest image 'largest': Download from the folder with the largest image
@ -199,11 +195,17 @@ Usage: sldl <input> [OPTIONS]
--album-art-only Only download album art for the provided album --album-art-only Only download album art for the provided album
--no-browse-folder Do not automatically browse user shares to get all files in --no-browse-folder Do not automatically browse user shares to get all files in
in the folder in the folder
--failed-album-path Path to move all album files to when one of the items from
the directory fails to download. Set to 'delete' to delete
the files instead. Set to 'disable' keep it where it is.
Default: {configured output dir}/failed
``` ```
``` ```
Aggregate Download Aggregate Download
-g, --aggregate Aggregate download mode: Find and download all distinct -g, --aggregate Aggregate download mode: Find and download all distinct
songs associated with the provided artist, album, or title. songs associated with the provided artist, album, or title.
--aggregate-length-tol <tol> Max length tolerance in seconds to consider two tracks or
albums equal. (Default: 3)
--min-shares-aggregate <num> Minimum number of shares of a track or album for it to be --min-shares-aggregate <num> Minimum number of shares of a track or album for it to be
downloaded in aggregate mode. (Default: 2) downloaded in aggregate mode. (Default: 2)
--relax-filtering Slightly relax file filtering in aggregate mode to include --relax-filtering Slightly relax file filtering in aggregate mode to include
@ -226,7 +228,7 @@ Usage: sldl <input> [OPTIONS]
## Input types ## Input types
The input type is usually determined automatically. To force a specific input type, set The input type is usually determined automatically. To force a specific input type, set
--input-type [spotify|youtube|csv|string|bandcamp]. The following input types are available: --input-type [spotify|youtube|csv|string|bandcamp|list]. The following input types are available:
### CSV file ### CSV file
Path to a local CSV file: Use a csv file containing track info of the songs to download. Path to a local CSV file: Use a csv file containing track info of the songs to download.
@ -290,6 +292,7 @@ artist
album album
length (in seconds) length (in seconds)
artist-maybe-wrong artist-maybe-wrong
album-track-count
``` ```
Example inputs and their interpretations: Example inputs and their interpretations:
``` ```
@ -302,6 +305,17 @@ Input String | Artist | Title | Album | Lengt
'artist=AR, title=T, album=AL' | AR | T | AL | 'artist=AR, title=T, album=AL' | AR | T | AL |
``` ```
### List
A path to a text file where each line has the following form:
```
"some input" "conditions" "preferred conditions"
"album=Album" "format=mp3; br > 128" "br >= 320"
```
Where "some input" is any of the above input types. The quotes can be omitted if the field
contains no spaces. The conditions and preferred conditions fields are added on top of the
configured conditions and can also be omitted. List input must be manually activated with
--input-type=list.
## Download modes ## Download modes
### Normal ### Normal
@ -313,24 +327,18 @@ Input String | Artist | Title | Album | Lengt
string or csv row has no track title, or when -a/--album is enabled. string or csv row has no track title, or when -a/--album is enabled.
### Aggregate ### Aggregate
With -g/--aggregate, sldl will first perform an ordinary search for the input, then attempt to With -g/--aggregate, sldl performs an ordinary search for the input then attempts to
group the results into distinct songs and download one of each kind. A common use case is group the results into distinct songs and download one of each kind, starting with the one
finding all remixes of a song or printing all songs by an artist that are not your music dir. which is shared by the most users.
Two files are considered equal if their inferred track title and artist name are equal Note that --min-shares-aggregate is 2 by default, which means that songs shared by only
(ignoring case and some special characters), and their lengths are within --length-tol of each one user will be ignored.
other.
Note that this mode is not 100% reliable, which is why --min-shares-aggregate is set to 2 by
default, i.e. any song that is shared only once will be ignored.
### Album Aggregate ### Album Aggregate
Activated when --album and --aggregate are enabled, in this mode sldl searches for the query Activated when both --album and --aggregate are enabled. sldl will group shares and download
and groups results into distinct albums. Two folders are considered same if they have the one of each distinct album, starting with the one shared by the most users. It's
same number of audio files, and the durations of the files are within --length-tol of each recommended to pair this with --interactive.
other (or within 3 seconds if length-tol is not configured). If both folders have exactly one Note that --min-shares-aggregate is 2 by default, which means that albums shared by only
audio file with similar lengths, also checks if the inferred title and artist name coincide. one user will be ignored.
More reliable than normal aggregate due to much simpler grouping logic.
Note that --min-shares-aggregate is 2 by default, which means that folders shared only once
will be ignored.
## Searching ## Searching
@ -407,7 +415,7 @@ only satisfies the format condition. Run with --print "results-full" to reveal t
ranking with this option due to the bitrate and samplerate checks. ranking with this option due to the bitrate and samplerate checks.
Conditions can also be supplied as a semicolon-delimited string with --cond and --pref, e.g Conditions can also be supplied as a semicolon-delimited string with --cond and --pref, e.g
--cond "br>=320;f=mp3,ogg;sr<96000" --cond "br >= 320; format = mp3,ogg; sr < 96000".
## Name format ## Name format
@ -443,7 +451,6 @@ track Track number
disc Disc number disc Disc number
filename Soulseek filename without extension filename Soulseek filename without extension
foldername Soulseek folder name foldername Soulseek folder name
default-foldername Default sldl folder name
extractor Name of the extractor used (CSV/Spotify/YouTube/etc) extractor Name of the extractor used (CSV/Spotify/YouTube/etc)
``` ```

149
slsk-batchdl/Config.cs
View file

@ -21,13 +21,10 @@ static class Config
AcceptNoLength = false, AcceptNoLength = false,
}; };
public static string parentFolder = Directory.GetCurrentDirectory(); public static string parentDir = Directory.GetCurrentDirectory();
public static string input = ""; public static string input = "";
public static string outputFolder = "";
public static string m3uFilePath = ""; public static string m3uFilePath = "";
public static string musicDir = ""; public static string musicDir = "";
public static string folderName = "";
public static string defaultFolderName = "";
public static string spotifyId = ""; public static string spotifyId = "";
public static string spotifySecret = ""; public static string spotifySecret = "";
public static string spotifyToken = ""; public static string spotifyToken = "";
@ -49,11 +46,11 @@ static class Config
public static string onComplete = ""; public static string onComplete = "";
public static string confPath = ""; public static string confPath = "";
public static string profile = ""; public static string profile = "";
public static string failedAlbumPath = "";
public static bool aggregate = false; public static bool aggregate = false;
public static bool album = false; public static bool album = false;
public static bool albumArtOnly = false; public static bool albumArtOnly = false;
public static bool interactiveMode = false; public static bool interactiveMode = false;
public static bool albumIgnoreFails = false;
public static bool setAlbumMinTrackCount = true; public static bool setAlbumMinTrackCount = true;
public static bool setAlbumMaxTrackCount = false; public static bool setAlbumMaxTrackCount = false;
public static bool skipNotFound = false; public static bool skipNotFound = false;
@ -94,6 +91,7 @@ static class Config
public static int listenPort = 49998; public static int listenPort = 49998;
public static int searchesPerTime = 34; public static int searchesPerTime = 34;
public static int searchRenewTime = 220; public static int searchRenewTime = 220;
public static int aggregateLengthTol = 3;
public static double fastSearchMinUpSpeed = 1.0; public static double fastSearchMinUpSpeed = 1.0;
public static Track regexToReplace = new(); public static Track regexToReplace = new();
public static Track regexReplaceBy = new(); public static Track regexReplaceBy = new();
@ -105,19 +103,22 @@ static class Config
public static SkipMode skipModeMusicDir = SkipMode.Name; public static SkipMode skipModeMusicDir = SkipMode.Name;
public static PrintOption printOption = PrintOption.None; public static PrintOption printOption = PrintOption.None;
static readonly Dictionary<string, (List<string> args, string? cond)> profiles = new();
static readonly HashSet<string> appliedProfiles = new();
static bool hasConfiguredM3uMode = false;
static bool confPathChanged = false;
static string[] arguments;
public static bool HasAutoProfiles { get; private set; } = false; public static bool HasAutoProfiles { get; private set; } = false;
public static bool DoNotDownload => (printOption & (PrintOption.Results | PrintOption.Tracks)) != 0; public static bool DoNotDownload => (printOption & (PrintOption.Results | PrintOption.Tracks)) != 0;
public static bool PrintTracks => (printOption & PrintOption.Tracks) != 0; public static bool PrintTracks => (printOption & PrintOption.Tracks) != 0;
public static bool PrintResults => (printOption & PrintOption.Results) != 0; public static bool PrintResults => (printOption & PrintOption.Results) != 0;
public static bool PrintTracksFull => (printOption & PrintOption.Tracks) != 0 && (printOption & PrintOption.Full) != 0; public static bool PrintTracksFull => (printOption & PrintOption.Tracks) != 0 && (printOption & PrintOption.Full) != 0;
public static bool PrintResultsFull => (printOption & PrintOption.Results) != 0 && (printOption & PrintOption.Full) != 0; public static bool PrintResultsFull => (printOption & PrintOption.Results) != 0 && (printOption & PrintOption.Full) != 0;
public static bool DeleteAlbumOnFail => failedAlbumPath == "delete";
public static bool IgnoreAlbumFail => failedAlbumPath == "disable";
static readonly Dictionary<string, (List<string> args, string? cond)> profiles = new();
static readonly HashSet<string> appliedProfiles = new();
static bool hasConfiguredM3uMode = false;
static bool confPathChanged = false;
static string[] arguments;
static FileConditions? prevConds = null;
static FileConditions? prevPrefConds = null;
public static bool ParseArgsAndReadConfig(string[] args) public static bool ParseArgsAndReadConfig(string[] args)
{ {
@ -218,7 +219,7 @@ static class Config
} }
public static void PostProcessArgs() // must be run after Program.trackLists has been assigned public static void PostProcessArgs()
{ {
if (DoNotDownload || debugInfo) if (DoNotDownload || debugInfo)
concurrentProcesses = 1; concurrentProcesses = 1;
@ -227,38 +228,21 @@ static class Config
if (DoNotDownload) if (DoNotDownload)
m3uOption = M3uOption.None; m3uOption = M3uOption.None;
else if (!hasConfiguredM3uMode) else if (!hasConfiguredM3uMode && inputType == InputType.String)
{
if (inputType == InputType.String)
m3uOption = M3uOption.None; m3uOption = M3uOption.None;
else if (!aggregate && !(skipExisting && (skipMode == SkipMode.M3u || skipMode == SkipMode.M3uCond))
&& Program.trackLists != null && !Program.trackLists.Flattened(true, false, true).Skip(1).Any())
{
m3uOption = M3uOption.None;
}
}
if (albumArtOnly && albumArtOption == AlbumArtOption.Default) if (albumArtOnly && albumArtOption == AlbumArtOption.Default)
albumArtOption = AlbumArtOption.Largest; albumArtOption = AlbumArtOption.Largest;
parentFolder = Utils.ExpandUser(parentFolder);
m3uFilePath = Utils.ExpandUser(m3uFilePath);
musicDir = Utils.ExpandUser(musicDir);
if (folderName.Length == 0)
folderName = defaultFolderName;
if (folderName == ".")
folderName = "";
folderName = folderName.Replace('\\', '/');
folderName = string.Join('/', folderName.Split('/').Select(x => x.ReplaceInvalidChars(invalidReplaceStr).Trim()));
folderName = folderName.Replace('/', Path.DirectorySeparatorChar);
outputFolder = Path.Join(parentFolder, folderName);
nameFormat = nameFormat.Trim(); nameFormat = nameFormat.Trim();
if (m3uFilePath.Length == 0) parentDir = Utils.ExpandUser(parentDir);
m3uFilePath = Path.Join(outputFolder, (folderName.Length == 0 ? "playlist" : folderName) + ".m3u8"); m3uFilePath = Utils.ExpandUser(m3uFilePath);
musicDir = Utils.ExpandUser(musicDir);
failedAlbumPath = Utils.ExpandUser(failedAlbumPath);
if (failedAlbumPath.Length == 0)
failedAlbumPath = Path.Join(parentDir, "failed");
} }
@ -313,7 +297,7 @@ static class Config
} }
public static void UpdateArgs(TrackListEntry tle) public static void UpdateProfiles(TrackListEntry tle)
{ {
if (DoNotDownload) if (DoNotDownload)
return; return;
@ -491,9 +475,32 @@ static class Config
} }
static void ParseConditions(FileConditions cond, string input) public static void AddTemporaryConditions(FileConditionsPatch? cond, FileConditionsPatch? prefCond)
{ {
static void UpdateMinMax(string value, string condition, ref int min, ref int max) if (cond != null)
{
prevConds = necessaryCond;
necessaryCond = necessaryCond.With(cond);
}
if (prefCond != null)
{
prevPrefConds = preferredCond;
preferredCond = preferredCond.With(prefCond);
}
}
public static void RestoreConditions()
{
if (prevConds != null)
necessaryCond = prevConds;
if (prevPrefConds != null)
preferredCond = prevPrefConds;
}
public static FileConditionsPatch ParseConditions(string input)
{
static void UpdateMinMax(string value, string condition, ref int? min, ref int? max)
{ {
if (condition.Contains(">=")) if (condition.Contains(">="))
min = int.Parse(value); min = int.Parse(value);
@ -507,6 +514,8 @@ static class Config
min = max = int.Parse(value); min = max = int.Parse(value);
} }
var cond = new FileConditionsPatch();
var tr = StringSplitOptions.TrimEntries | StringSplitOptions.RemoveEmptyEntries; var tr = StringSplitOptions.TrimEntries | StringSplitOptions.RemoveEmptyEntries;
string[] conditions = input.Split(';', tr); string[] conditions = input.Split(';', tr);
foreach (string condition in conditions) foreach (string condition in conditions)
@ -569,6 +578,8 @@ static class Config
throw new ArgumentException($"Unknown condition '{condition}'"); throw new ArgumentException($"Unknown condition '{condition}'");
} }
} }
return cond;
} }
@ -614,21 +625,18 @@ static class Config
"spotify" => InputType.Spotify, "spotify" => InputType.Spotify,
"bandcamp" => InputType.Bandcamp, "bandcamp" => InputType.Bandcamp,
"string" => InputType.String, "string" => InputType.String,
"list" => InputType.List,
_ => throw new ArgumentException($"Invalid input type '{args[i]}'"), _ => throw new ArgumentException($"Invalid input type '{args[i]}'"),
}; };
break; break;
case "-p": case "-p":
case "--path": case "--path":
parentFolder = args[++i]; parentDir = args[++i];
break; break;
case "-c": case "-c":
case "--config": case "--config":
confPath = args[++i]; confPath = args[++i];
break; break;
case "-f":
case "--folder":
folderName = args[++i];
break;
case "-m": case "-m":
case "--md": case "--md":
case "--music-dir": case "--music-dir":
@ -942,9 +950,9 @@ static class Config
preferredCond = new FileConditions(); preferredCond = new FileConditions();
necessaryCond = new FileConditions(); necessaryCond = new FileConditions();
break; break;
case "--aif": case "--fap":
case "--album-ignore-fails": case "--failed-album-path":
setFlag(ref albumIgnoreFails, ref i); failedAlbumPath = args[++i];
break; break;
case "-t": case "-t":
case "--interactive": case "--interactive":
@ -1063,12 +1071,12 @@ static class Config
case "--c": case "--c":
case "--cond": case "--cond":
case "--conditions": case "--conditions":
ParseConditions(necessaryCond, args[++i]); necessaryCond.With(ParseConditions(args[++i]));
break; break;
case "--pc": case "--pc":
case "--pref": case "--pref":
case "--preferred-conditions": case "--preferred-conditions":
ParseConditions(preferredCond, args[++i]); preferredCond.With(ParseConditions(args[++i]));
break; break;
case "--nmsc": case "--nmsc":
case "--no-modify-share-count": case "--no-modify-share-count":
@ -1177,6 +1185,10 @@ static class Config
case "--skip-existing-pref-cond": case "--skip-existing-pref-cond":
setFlag(ref skipExistingPrefCond, ref i); setFlag(ref skipExistingPrefCond, ref i);
break; break;
case "--alt":
case "--aggregate-length-tol":
aggregateLengthTol = int.Parse(args[++i]);
break;
default: default:
throw new ArgumentException($"Unknown argument: {args[i]}"); throw new ArgumentException($"Unknown argument: {args[i]}");
} }
@ -1193,4 +1205,41 @@ static class Config
} }
} }
} }
public static string[] GetArgsArray(string commandLine)
{
var args = new List<string>();
var currentArg = new StringBuilder();
bool inQuotes = false;
for (int i = 0; i < commandLine.Length; i++)
{
char c = commandLine[i];
if (c == '\"')
{
inQuotes = !inQuotes;
}
else if (c == ' ' && !inQuotes)
{
if (currentArg.Length > 0)
{
args.Add(currentArg.ToString());
currentArg.Clear();
}
}
else
{
currentArg.Append(c);
}
}
if (currentArg.Length > 0)
{
args.Add(currentArg.ToString());
}
return args.ToArray();
}
} }

70
slsk-batchdl/Data.cs
View file

@ -100,38 +100,51 @@ namespace Data
public class TrackListEntry public class TrackListEntry
{ {
public List<List<Track>> list; public List<List<Track>>? list;
public Track source; public Track source;
public bool needSourceSearch = false; public bool needSourceSearch = false;
public bool sourceCanBeSkipped = false; public bool sourceCanBeSkipped = false;
public bool needSkipExistingAfterSearch = false; public bool needSkipExistingAfterSearch = false;
public bool gotoNextAfterSearch = false; public bool gotoNextAfterSearch = false;
public bool placeInSubdir = false; public string? defaultFolderName = null;
public FileConditionsPatch? additionalConds = null;
public FileConditionsPatch? additionalPrefConds = null;
public TrackListEntry() public TrackListEntry(TrackType trackType)
{ {
list = new List<List<Track>>(); list = new List<List<Track>>();
source = new Track(); this.source = new Track() { Type = trackType };
SetDefaults();
} }
public TrackListEntry(Track source) public TrackListEntry(Track source)
{ {
list = new List<List<Track>>(); list = new List<List<Track>>();
this.source = source; this.source = source;
SetDefaults();
needSourceSearch = source.Type != TrackType.Normal;
needSkipExistingAfterSearch = source.Type == TrackType.Aggregate;
gotoNextAfterSearch = source.Type == TrackType.AlbumAggregate;
sourceCanBeSkipped = source.Type != TrackType.Normal
&& source.Type != TrackType.Aggregate
&& source.Type != TrackType.AlbumAggregate;
} }
public TrackListEntry(List<List<Track>> list, Track source) public TrackListEntry(List<List<Track>> list, Track source)
{ {
this.list = list; this.list = list;
this.source = source; this.source = source;
SetDefaults();
}
public TrackListEntry(List<List<Track>> list, Track source, bool needSourceSearch = false, bool sourceCanBeSkipped = false,
bool needSkipExistingAfterSearch = false, bool gotoNextAfterSearch = false, string? defaultFoldername = null)
{
this.list = list;
this.source = source;
this.needSourceSearch = needSourceSearch;
this.sourceCanBeSkipped = sourceCanBeSkipped;
this.needSkipExistingAfterSearch = needSkipExistingAfterSearch;
this.gotoNextAfterSearch = gotoNextAfterSearch;
this.defaultFolderName = defaultFoldername;
}
public void SetDefaults()
{
needSourceSearch = source.Type != TrackType.Normal; needSourceSearch = source.Type != TrackType.Normal;
needSkipExistingAfterSearch = source.Type == TrackType.Aggregate; needSkipExistingAfterSearch = source.Type == TrackType.Aggregate;
gotoNextAfterSearch = source.Type == TrackType.AlbumAggregate; gotoNextAfterSearch = source.Type == TrackType.AlbumAggregate;
@ -140,16 +153,14 @@ namespace Data
&& source.Type != TrackType.AlbumAggregate; && source.Type != TrackType.AlbumAggregate;
} }
public TrackListEntry(List<List<Track>> list, Track source, bool needSearch, bool placeInSubdir, public void AddTrack(Track track)
bool sourceCanBeSkipped, bool needSkipExistingAfterSearch, bool gotoNextAfterSearch)
{ {
this.list = list; if (list == null)
this.source = source; list = new List<List<Track>>() { new List<Track>() { track } };
this.needSourceSearch = needSearch; else if (list.Count == 0)
this.placeInSubdir = placeInSubdir; list.Add(new List<Track>() { track });
this.sourceCanBeSkipped = sourceCanBeSkipped; else
this.needSkipExistingAfterSearch = needSkipExistingAfterSearch; list[0].Add(track);
this.gotoNextAfterSearch = gotoNextAfterSearch;
} }
} }
@ -174,7 +185,7 @@ namespace Data
} }
else else
{ {
res.AddEntry(new TrackListEntry()); res.AddEntry(new TrackListEntry(TrackType.Normal));
res.AddTrackToLast(track); res.AddTrackToLast(track);
bool hasNext; bool hasNext;
@ -252,11 +263,13 @@ namespace Data
if (tle.source.Type == TrackType.Album && aggregate) if (tle.source.Type == TrackType.Album && aggregate)
{ {
tle.source.Type = TrackType.AlbumAggregate; tle.source.Type = TrackType.AlbumAggregate;
tle.SetDefaults();
newLists.Add(tle); newLists.Add(tle);
} }
else if (tle.source.Type == TrackType.Aggregate && album) else if (tle.source.Type == TrackType.Aggregate && album)
{ {
tle.source.Type = TrackType.AlbumAggregate; tle.source.Type = TrackType.AlbumAggregate;
tle.SetDefaults();
newLists.Add(tle); newLists.Add(tle);
} }
else if (tle.source.Type == TrackType.Normal && (album || aggregate)) else if (tle.source.Type == TrackType.Normal && (album || aggregate))
@ -270,7 +283,9 @@ namespace Data
else if (aggregate) else if (aggregate)
track.Type = TrackType.Aggregate; track.Type = TrackType.Aggregate;
newLists.Add(new TrackListEntry(track)); var newTle = new TrackListEntry(track);
newTle.defaultFolderName = tle.defaultFolderName;
newLists.Add(newTle);
} }
} }
else else
@ -284,16 +299,11 @@ namespace Data
public void SetListEntryOptions() public void SetListEntryOptions()
{ {
// place downloads in subdirs if there is more than one special (album/aggregate) download // aggregate downloads will be placed in subfolders by default
bool placeInSubdirs = Flattened(true, false, true).Skip(1).Any();
if (placeInSubdirs)
{
foreach (var tle in lists) foreach (var tle in lists)
{ {
if (tle.source.Type != TrackType.Normal) if (tle.source.Type == TrackType.Aggregate || tle.source.Type == TrackType.AlbumAggregate)
tle.placeInSubdir = true; tle.defaultFolderName = Path.Join(tle.defaultFolderName, tle.source.ToString(true));
}
} }
} }

16
slsk-batchdl/Download.cs
View file

@ -17,7 +17,7 @@ using SlDictionary = System.Collections.Concurrent.ConcurrentDictionary<string,
static class Download static class Download
{ {
public static async Task DownloadFile(SearchResponse response, Soulseek.File file, string filePath, Track track, ProgressBar progress, CancellationTokenSource? searchCts = null) public static async Task DownloadFile(SearchResponse response, Soulseek.File file, string filePath, Track track, ProgressBar progress, CancellationTokenSource cts, CancellationTokenSource? searchCts = null)
{ {
if (Config.DoNotDownload) if (Config.DoNotDownload)
throw new Exception(); throw new Exception();
@ -42,16 +42,10 @@ static class Download
try try
{ {
using var cts = new CancellationTokenSource();
using var outputStream = new FileStream(filePath, FileMode.Create); using var outputStream = new FileStream(filePath, FileMode.Create);
var wrapper = new DownloadWrapper(origPath, response, file, track, cts, progress); var wrapper = new DownloadWrapper(origPath, response, file, track, cts, progress);
downloads.TryAdd(file.Filename, wrapper); downloads.TryAdd(file.Filename, wrapper);
// Attempt to make it resume downloads after a network interruption.
// Does not work: The resumed download will be queued until it goes stale.
// The host (slskd) reports that "Another upload to {user} is already in progress"
// when attempting to resume. Must wait until timeout, which can take minutes.
int maxRetries = 3; int maxRetries = 3;
int retryCount = 0; int retryCount = 0;
while (true) while (true)
@ -65,10 +59,12 @@ static class Download
break; break;
} }
catch (SoulseekClientException) catch (SoulseekClientException e)
{ {
retryCount++; retryCount++;
Printing.WriteLine($"Error while downloading: {e}", ConsoleColor.DarkYellow, debugOnly: true);
if (retryCount >= maxRetries || IsConnectedAndLoggedIn()) if (retryCount >= maxRetries || IsConnectedAndLoggedIn())
throw; throw;
@ -86,13 +82,13 @@ static class Download
throw; throw;
} }
try { searchCts?.Cancel(); } try { searchCts?.Cancel(); } catch { }
catch { }
try { Utils.Move(filePath, origPath); } try { Utils.Move(filePath, origPath); }
catch (IOException) { Printing.WriteLine($"Failed to rename .incomplete file", ConsoleColor.DarkYellow, true); } catch (IOException) { Printing.WriteLine($"Failed to rename .incomplete file", ConsoleColor.DarkYellow, true); }
downloads.TryRemove(file.Filename, out var x); downloads.TryRemove(file.Filename, out var x);
if (x != null) if (x != null)
{ {
lock (x) lock (x)

8
slsk-batchdl/Enums.cs
View file

@ -37,6 +37,7 @@ namespace Enums
Spotify, Spotify,
Bandcamp, Bandcamp,
String, String,
List,
None, None,
} }
@ -85,4 +86,11 @@ namespace Enums
Normal, Normal,
Verbose Verbose
} }
public enum AlbumFailOption
{
Ignore,
Keep,
Delete,
}
} }

25
slsk-batchdl/Extractors/Bandcamp.cs
View file

@ -16,17 +16,17 @@ namespace Extractors
return input.IsInternetUrl() && input.Contains("bandcamp.com"); return input.IsInternetUrl() && input.Contains("bandcamp.com");
} }
public async Task<TrackLists> GetTracks(int maxTracks, int offset, bool reverse) public async Task<TrackLists> GetTracks(string input, int maxTracks, int offset, bool reverse)
{ {
var trackLists = new TrackLists(); var trackLists = new TrackLists();
bool isTrack = Config.input.Contains("/track/"); bool isTrack = input.Contains("/track/");
bool isAlbum = !isTrack && Config.input.Contains("/album/"); bool isAlbum = !isTrack && input.Contains("/album/");
bool isArtist =!isTrack && !isAlbum; bool isArtist =!isTrack && !isAlbum;
if (isArtist) if (isArtist)
{ {
Console.WriteLine("Retrieving bandcamp artist discography.."); Console.WriteLine("Retrieving bandcamp artist discography..");
string artistUrl = Config.input.TrimEnd('/'); string artistUrl = input.TrimEnd('/');
if (!artistUrl.EndsWith("/music")) if (!artistUrl.EndsWith("/music"))
artistUrl += "/music"; artistUrl += "/music";
@ -51,17 +51,14 @@ namespace Extractors
foreach (var item in root.GetProperty("discography").EnumerateArray()) foreach (var item in root.GetProperty("discography").EnumerateArray())
{ {
//ItemType = item.GetProperty("item_type").GetString(), //ItemType = item.GetProperty("item_type").GetString(),
var t = new Track() var track = new Track()
{ {
Album = item.GetProperty("title").GetString(), Album = item.GetProperty("title").GetString(),
Artist = item.GetProperty("artist_name").GetString() ?? item.GetProperty("band_name").GetString(), Artist = item.GetProperty("artist_name").GetString() ?? item.GetProperty("band_name").GetString(),
Type = TrackType.Album, Type = TrackType.Album,
}; };
var tle = new TrackListEntry() var tle = new TrackListEntry(track);
{ tle.defaultFolderName = track.Artist;
source = t,
placeInSubdir = true,
};
trackLists.AddEntry(tle); trackLists.AddEntry(tle);
} }
} }
@ -69,7 +66,7 @@ namespace Extractors
{ {
Console.WriteLine("Retrieving bandcamp item.."); Console.WriteLine("Retrieving bandcamp item..");
var web = new HtmlWeb(); var web = new HtmlWeb();
var doc = await web.LoadFromWebAsync(Config.input); var doc = await web.LoadFromWebAsync(input);
var nameSection = doc.DocumentNode.SelectSingleNode("//div[@id='name-section']"); var nameSection = doc.DocumentNode.SelectSingleNode("//div[@id='name-section']");
var name = nameSection.SelectSingleNode(".//h2[contains(@class, 'trackTitle')]").InnerText.UnHtmlString().Trim(); var name = nameSection.SelectSingleNode(".//h2[contains(@class, 'trackTitle')]").InnerText.UnHtmlString().Trim();
@ -91,8 +88,6 @@ namespace Extractors
if (Config.setAlbumMaxTrackCount) if (Config.setAlbumMaxTrackCount)
track.MaxAlbumTrackCount = n; track.MaxAlbumTrackCount = n;
} }
Config.defaultFolderName = track.ToString(true).ReplaceInvalidChars(Config.invalidReplaceStr).Trim();
} }
else else
{ {
@ -101,10 +96,8 @@ namespace Extractors
//var timeParts = doc.DocumentNode.SelectSingleNode("//span[@class='time_total']").InnerText.Trim().Split(':'); //var timeParts = doc.DocumentNode.SelectSingleNode("//span[@class='time_total']").InnerText.Trim().Split(':');
var track = new Track() { Artist = artist, Title = name, Album = album }; var track = new Track() { Artist = artist, Title = name, Album = album };
trackLists.AddEntry(new()); trackLists.AddEntry(new TrackListEntry(TrackType.Normal));
trackLists.AddTrackToLast(track); trackLists.AddTrackToLast(track);
Config.defaultFolderName = ".";
} }
} }

28
slsk-batchdl/Extractors/Csv.cs
View file

@ -6,7 +6,8 @@ namespace Extractors
{ {
public class CsvExtractor : IExtractor public class CsvExtractor : IExtractor
{ {
object csvLock = new(); string? csvFilePath = null;
readonly object csvLock = new();
int csvColumnCount = -1; int csvColumnCount = -1;
public static bool InputMatches(string input) public static bool InputMatches(string input)
@ -15,28 +16,26 @@ namespace Extractors
return !input.IsInternetUrl() && input.EndsWith(".csv"); return !input.IsInternetUrl() && input.EndsWith(".csv");
} }
public async Task<TrackLists> GetTracks(int maxTracks, int offset, bool reverse) public async Task<TrackLists> GetTracks(string input, int maxTracks, int offset, bool reverse)
{ {
if (!File.Exists(Config.input)) if (!File.Exists(input))
throw new FileNotFoundException("CSV file not found"); throw new FileNotFoundException("CSV file not found");
var tracks = await ParseCsvIntoTrackInfo(Config.input, Config.artistCol, Config.trackCol, Config.lengthCol, csvFilePath = input;
var tracks = await ParseCsvIntoTrackInfo(input, Config.artistCol, Config.trackCol, Config.lengthCol,
Config.albumCol, Config.descCol, Config.ytIdCol, Config.trackCountCol, Config.timeUnit, Config.ytParse); Config.albumCol, Config.descCol, Config.ytIdCol, Config.trackCountCol, Config.timeUnit, Config.ytParse);
if (reverse) if (reverse)
tracks.Reverse(); tracks.Reverse();
var trackLists = TrackLists.FromFlattened(tracks.Skip(offset).Take(maxTracks)); var trackLists = TrackLists.FromFlattened(tracks.Skip(offset).Take(maxTracks));
var csvName = Path.GetFileNameWithoutExtension(input);
foreach (var tle in trackLists.lists) foreach (var tle in trackLists.lists)
{ {
if (tle.source.Type != TrackType.Normal) tle.defaultFolderName = csvName;
{
tle.placeInSubdir = true;
} }
}
Config.defaultFolderName = Path.GetFileNameWithoutExtension(Config.input);
return trackLists; return trackLists;
} }
@ -45,16 +44,16 @@ namespace Extractors
{ {
lock (csvLock) lock (csvLock)
{ {
if (File.Exists(Config.input)) if (File.Exists(csvFilePath))
{ {
try try
{ {
string[] lines = File.ReadAllLines(Config.input, System.Text.Encoding.UTF8); string[] lines = File.ReadAllLines(csvFilePath, System.Text.Encoding.UTF8);
if (track.CsvRow > -1 && track.CsvRow < lines.Length) if (track.CsvRow > -1 && track.CsvRow < lines.Length)
{ {
lines[track.CsvRow] = new string(',', Math.Max(0, csvColumnCount - 1)); lines[track.CsvRow] = new string(',', Math.Max(0, csvColumnCount - 1));
Utils.WriteAllLines(Config.input, lines, '\n'); Utils.WriteAllLines(csvFilePath, lines, '\n');
} }
} }
catch (Exception e) catch (Exception e)
@ -193,7 +192,7 @@ namespace Extractors
return tracks; return tracks;
} }
double ParseTrackLength(string duration, string format) static double ParseTrackLength(string duration, string format)
{ {
if (string.IsNullOrEmpty(format)) if (string.IsNullOrEmpty(format))
throw new ArgumentException("Duration format string empty"); throw new ArgumentException("Duration format string empty");
@ -224,6 +223,5 @@ namespace Extractors
return totalSeconds; return totalSeconds;
} }
} }
} }

146
slsk-batchdl/Extractors/List.cs Normal file
View file

@ -0,0 +1,146 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Data;
using Enums;
namespace Extractors
{
public class ListExtractor : IExtractor
{
string? listFilePath = null;
readonly object fileLock = new object();
public static bool InputMatches(string input)
{
return !input.IsInternetUrl();
}
public async Task<TrackLists> GetTracks(string input, int maxTracks, int offset, bool reverse)
{
if (!File.Exists(input))
throw new FileNotFoundException("List file not found");
listFilePath = input;
var lines = File.ReadAllLines(input);
var trackLists = new TrackLists();
int step = reverse ? -1 : 1;
int start = reverse ? lines.Length - 1 : 0;
int count = 0;
int added = 0;
string foldername = Path.GetFileNameWithoutExtension(input);
for (int i = start; i < lines.Length && i >= 0; i += step)
{
var line = lines[i].Trim();
if (line.Length == 0 || line.StartsWith('#')) continue;
if (count++ < offset)
continue;
if (added >= maxTracks)
break;
var fields = ParseLine(line);
var (_, ex) = ExtractorRegistry.GetMatchingExtractor(fields[0]);
var tl = await ex.GetTracks(fields[0], int.MaxValue, 0, false);
foreach (var tle in tl.lists)
{
if (fields.Count >= 2)
tle.additionalConds = Config.ParseConditions(fields[1]);
if (fields.Count >= 3)
tle.additionalPrefConds = Config.ParseConditions(fields[2]);
tle.defaultFolderName = foldername;
}
if (tl.lists.Count == 1)
tl[0].source.CsvRow = i;
trackLists.lists.AddRange(tl.lists);
added++;
}
return trackLists;
}
static List<string> ParseLine(string input)
{
var fields = new List<string>();
bool inQuotes = false;
var currentField = new StringBuilder();
input = input.Replace('\t', ' ');
for (int i = 0; i < input.Length; i++)
{
char c = input[i];
if (c == '"')
{
inQuotes = !inQuotes;
}
else if (c == ' ' && !inQuotes)
{
if (currentField.Length > 0)
{
fields.Add(currentField.ToString());
currentField.Clear();
}
while (i < input.Length - 1 && input[i + 1] == ' ')
{
i++;
}
}
else
{
currentField.Append(c);
}
}
if (currentField.Length > 0)
{
fields.Add(currentField.ToString());
}
return fields;
}
public async Task RemoveTrackFromSource(Track track)
{
lock (fileLock)
{
if (File.Exists(listFilePath))
{
try
{
string[] lines = File.ReadAllLines(listFilePath, Encoding.UTF8);
if (track.CsvRow > -1 && track.CsvRow < lines.Length)
{
lines[track.CsvRow] = "";
Utils.WriteAllLines(listFilePath, lines, '\n');
}
}
catch (Exception e)
{
Printing.WriteLine($"Error removing from source: {e}", debugOnly: true);
}
}
}
}
}
}

34
slsk-batchdl/Extractors/Spotify.cs
View file

@ -19,37 +19,35 @@ namespace Extractors
return input == "spotify-likes" || input.IsInternetUrl() && input.Contains("spotify.com"); return input == "spotify-likes" || input.IsInternetUrl() && input.Contains("spotify.com");
} }
public async Task<TrackLists> GetTracks(int maxTracks, int offset, bool reverse) public async Task<TrackLists> GetTracks(string input, int maxTracks, int offset, bool reverse)
{ {
var trackLists = new TrackLists(); var trackLists = new TrackLists();
int max = reverse ? int.MaxValue : maxTracks; int max = reverse ? int.MaxValue : maxTracks;
int off = reverse ? 0 : offset; int off = reverse ? 0 : offset;
string playlistName = ""; bool needLogin = input == "spotify-likes" || Config.removeTracksFromSource;
bool needLogin = Config.input == "spotify-likes" || Config.removeTracksFromSource; var tle = new TrackListEntry(TrackType.Normal);
var tle = new TrackListEntry();
if (needLogin && Config.spotifyToken.Length == 0 && (Config.spotifyId.Length == 0 || Config.spotifySecret.Length == 0)) if (needLogin && Config.spotifyToken.Length == 0 && (Config.spotifyId.Length == 0 || Config.spotifySecret.Length == 0))
{ {
Console.WriteLine("Error: Credentials are required when downloading liked music or removing from source playlists."); Console.WriteLine("Error: Credentials are required when downloading liked music or removing from source playlists.");
Environment.Exit(0); Environment.Exit(1);
} }
spotifyClient = new Spotify(Config.spotifyId, Config.spotifySecret, Config.spotifyToken, Config.spotifyRefresh); spotifyClient = new Spotify(Config.spotifyId, Config.spotifySecret, Config.spotifyToken, Config.spotifyRefresh);
await spotifyClient.Authorize(needLogin, Config.removeTracksFromSource); await spotifyClient.Authorize(needLogin, Config.removeTracksFromSource);
if (Config.input == "spotify-likes") if (input == "spotify-likes")
{ {
Console.WriteLine("Loading Spotify likes.."); Console.WriteLine("Loading Spotify likes..");
var tracks = await spotifyClient.GetLikes(max, off); var tracks = await spotifyClient.GetLikes(max, off);
playlistName = "Spotify Likes"; tle.defaultFolderName = "Spotify Likes";
tle.list.Add(tracks); tle.list.Add(tracks);
} }
else if (Config.input.Contains("/album/")) else if (input.Contains("/album/"))
{ {
Console.WriteLine("Loading Spotify album.."); Console.WriteLine("Loading Spotify album..");
(var source, var tracks) = await spotifyClient.GetAlbum(Config.input); (var source, var tracks) = await spotifyClient.GetAlbum(input);
playlistName = source.ToString(noInfo: true);
tle.source = source; tle.source = source;
if (Config.setAlbumMinTrackCount) if (Config.setAlbumMinTrackCount)
@ -58,11 +56,11 @@ namespace Extractors
if (Config.setAlbumMaxTrackCount) if (Config.setAlbumMaxTrackCount)
source.MaxAlbumTrackCount = tracks.Count; source.MaxAlbumTrackCount = tracks.Count;
} }
else if (Config.input.Contains("/artist/")) else if (input.Contains("/artist/"))
{ {
Console.WriteLine("Loading spotify artist.."); Console.WriteLine("Loading spotify artist..");
Console.WriteLine("Error: Spotify artist download currently not supported."); Console.WriteLine("Error: Spotify artist download currently not supported.");
Environment.Exit(0); Environment.Exit(1);
} }
else else
{ {
@ -71,19 +69,21 @@ namespace Extractors
try try
{ {
Console.WriteLine("Loading Spotify playlist"); Console.WriteLine("Loading Spotify playlist");
(playlistName, playlistUri, tracks) = await spotifyClient.GetPlaylist(Config.input, max, off); (var playlistName, playlistUri, tracks) = await spotifyClient.GetPlaylist(input, max, off);
tle.defaultFolderName = playlistName;
} }
catch (SpotifyAPI.Web.APIException) catch (SpotifyAPI.Web.APIException)
{ {
if (!needLogin && !spotifyClient.UsedDefaultCredentials) if (!needLogin && !spotifyClient.UsedDefaultCredentials)
{ {
await spotifyClient.Authorize(true, Config.removeTracksFromSource); await spotifyClient.Authorize(true, Config.removeTracksFromSource);
(playlistName, playlistUri, tracks) = await spotifyClient.GetPlaylist(Config.input, max, off); (var playlistName, playlistUri, tracks) = await spotifyClient.GetPlaylist(input, max, off);
tle.defaultFolderName = playlistName;
} }
else if (!needLogin) else if (!needLogin)
{ {
Console.WriteLine("Spotify playlist not found (it may be set to private, but no credentials have been provided)."); Console.WriteLine("Error: Spotify playlist not found (it may be set to private, but no credentials have been provided).");
Environment.Exit(0); Environment.Exit(1);
} }
else throw; else throw;
} }
@ -91,8 +91,6 @@ namespace Extractors
tle.list.Add(tracks); tle.list.Add(tracks);
} }
Config.defaultFolderName = playlistName.ReplaceInvalidChars(Config.invalidReplaceStr);
trackLists.AddEntry(tle); trackLists.AddEntry(tle);
if (reverse) if (reverse)

40
slsk-batchdl/Extractors/String.cs
View file

@ -11,35 +11,33 @@ namespace Extractors
return !input.IsInternetUrl(); return !input.IsInternetUrl();
} }
public async Task<TrackLists> GetTracks(int maxTracks, int offset, bool reverse) public async Task<TrackLists> GetTracks(string input, int maxTracks, int offset, bool reverse)
{ {
var trackLists = new TrackLists(); var trackLists = new TrackLists();
var music = ParseTrackArg(Config.input, Config.album); var music = ParseTrackArg(input, Config.album);
TrackListEntry tle;
if (Config.album || (music.Title.Length == 0 && music.Album.Length > 0)) if (Config.album || (music.Title.Length == 0 && music.Album.Length > 0))
{ {
music.Type = TrackType.Album; music.Type = TrackType.Album;
trackLists.AddEntry(new TrackListEntry(music)); tle = new TrackListEntry(music);
} }
else else
{ {
trackLists.AddEntry(new TrackListEntry()); tle = new TrackListEntry(TrackType.Normal);
trackLists.AddTrackToLast(music); tle.AddTrack(music);
} }
if (Config.aggregate || Config.album || music.Type != TrackType.Normal) trackLists.AddEntry(tle);
Config.defaultFolderName = music.ToString(true).ReplaceInvalidChars(Config.invalidReplaceStr).Trim();
else
Config.defaultFolderName = ".";
return trackLists; return trackLists;
} }
static public Track ParseTrackArg(string input, bool isAlbum) public static Track ParseTrackArg(string input, bool isAlbum)
{ {
input = input.Trim(); input = input.Trim();
var track = new Track(); var track = new Track();
var keys = new string[] { "title", "artist", "length", "album", "artist-maybe-wrong" }; var keys = new string[] { "title", "artist", "length", "album", "artist-maybe-wrong", "album-track-count" };
void setProperty(string key, string value) void setProperty(string key, string value)
{ {
@ -60,6 +58,26 @@ namespace Extractors
case "artist-maybe-wrong": case "artist-maybe-wrong":
if (value == "true") track.ArtistMaybeWrong = true; if (value == "true") track.ArtistMaybeWrong = true;
break; break;
case "album-track-count":
if (value == "-1")
{
track.MinAlbumTrackCount = -1;
track.MaxAlbumTrackCount = -1;
}
else if (value.Last() == '-')
{
track.MaxAlbumTrackCount = int.Parse(value[..^1]);
}
else if (value.Last() == '+')
{
track.MinAlbumTrackCount = int.Parse(value[..^1]);
}
else
{
track.MinAlbumTrackCount = int.Parse(value);
track.MaxAlbumTrackCount = track.MinAlbumTrackCount;
}
break;
} }
} }

20
slsk-batchdl/Extractors/YouTube.cs
View file

@ -9,6 +9,7 @@ using HtmlAgilityPack;
using System.Collections.Concurrent; using System.Collections.Concurrent;
using Data; using Data;
using Enums;
namespace Extractors namespace Extractors
{ {
@ -20,7 +21,7 @@ namespace Extractors
return input.IsInternetUrl() && (input.Contains("youtu.be") || input.Contains("youtube.com")); return input.IsInternetUrl() && (input.Contains("youtu.be") || input.Contains("youtube.com"));
} }
public async Task<TrackLists> GetTracks(int maxTracks, int offset, bool reverse) public async Task<TrackLists> GetTracks(string input, int maxTracks, int offset, bool reverse)
{ {
var trackLists = new TrackLists(); var trackLists = new TrackLists();
int max = reverse ? int.MaxValue : maxTracks; int max = reverse ? int.MaxValue : maxTracks;
@ -35,24 +36,24 @@ namespace Extractors
{ {
Console.WriteLine("Getting deleted videos.."); Console.WriteLine("Getting deleted videos..");
var archive = new YouTube.YouTubeArchiveRetriever(); var archive = new YouTube.YouTubeArchiveRetriever();
deleted = await archive.RetrieveDeleted(Config.input, printFailed: Config.deletedOnly); deleted = await archive.RetrieveDeleted(input, printFailed: Config.deletedOnly);
} }
if (!Config.deletedOnly) if (!Config.deletedOnly)
{ {
if (YouTube.apiKey.Length > 0) if (YouTube.apiKey.Length > 0)
{ {
Console.WriteLine("Loading YouTube playlist (API)"); Console.WriteLine("Loading YouTube playlist (API)");
(name, tracks) = await YouTube.GetTracksApi(Config.input, max, off); (name, tracks) = await YouTube.GetTracksApi(input, max, off);
} }
else else
{ {
Console.WriteLine("Loading YouTube playlist"); Console.WriteLine("Loading YouTube playlist");
(name, tracks) = await YouTube.GetTracksYtExplode(Config.input, max, off); (name, tracks) = await YouTube.GetTracksYtExplode(input, max, off);
} }
} }
else else
{ {
name = await YouTube.GetPlaylistTitle(Config.input); name = await YouTube.GetPlaylistTitle(input);
} }
if (deleted != null) if (deleted != null)
{ {
@ -61,11 +62,12 @@ namespace Extractors
YouTube.StopService(); YouTube.StopService();
var tle = new TrackListEntry(); var tle = new TrackListEntry(TrackType.Normal);
tle.list.Add(tracks);
trackLists.AddEntry(tle);
Config.defaultFolderName = name.ReplaceInvalidChars(Config.invalidReplaceStr); tle.defaultFolderName = name;
tle.list.Add(tracks);
trackLists.AddEntry(tle);
if (reverse) if (reverse)
{ {

21
slsk-batchdl/Extractors/_Extractor.cs
View file

@ -6,7 +6,7 @@ namespace Extractors
{ {
public interface IExtractor public interface IExtractor
{ {
Task<TrackLists> GetTracks(int maxTracks, int offset, bool reverse); Task<TrackLists> GetTracks(string input, int maxTracks, int offset, bool reverse);
Task RemoveTrackFromSource(Track track) => Task.CompletedTask; Task RemoveTrackFromSource(Track track) => Task.CompletedTask;
} }
@ -19,18 +19,29 @@ namespace Extractors
(InputType.Spotify, SpotifyExtractor.InputMatches, () => new SpotifyExtractor()), (InputType.Spotify, SpotifyExtractor.InputMatches, () => new SpotifyExtractor()),
(InputType.Bandcamp, BandcampExtractor.InputMatches, () => new BandcampExtractor()), (InputType.Bandcamp, BandcampExtractor.InputMatches, () => new BandcampExtractor()),
(InputType.String, StringExtractor.InputMatches, () => new StringExtractor()), (InputType.String, StringExtractor.InputMatches, () => new StringExtractor()),
(InputType.List, ListExtractor.InputMatches, () => new ListExtractor()),
}; };
public static (InputType, IExtractor?) GetMatchingExtractor(string input) public static (InputType, IExtractor) GetMatchingExtractor(string input, InputType inputType = InputType.None)
{ {
foreach ((var inputType, var inputMatches, var extractor) in extractors) if (string.IsNullOrEmpty(input))
throw new ArgumentException("Input string can not be null or empty.");
if (inputType != InputType.None)
{
var (t, _, e) = extractors.First(x => x.Item1 == inputType);
return (t, e());
}
foreach ((var type, var inputMatches, var extractor) in extractors)
{ {
if (inputMatches(input)) if (inputMatches(input))
{ {
return (inputType, extractor()); return (type, extractor());
} }
} }
return (InputType.None, null);
throw new ArgumentException($"No matching extractor for input '{input}'");
} }
} }
} }

73
slsk-batchdl/FileConditions.cs
View file

@ -41,6 +41,58 @@ public class FileConditions
BannedUsers = other.BannedUsers.ToArray(); BannedUsers = other.BannedUsers.ToArray();
} }
public FileConditions With(FileConditionsPatch patch)
{
var cond = new FileConditions(this);
if (patch.LengthTolerance != null)
cond.LengthTolerance = patch.LengthTolerance.Value;
if (patch.MinBitrate != null)
cond.MinBitrate = patch.MinBitrate.Value;
if (patch.MaxBitrate != null)
cond.MaxBitrate = patch.MaxBitrate.Value;
if (patch.MinSampleRate != null)
cond.MinSampleRate = patch.MinSampleRate.Value;
if (patch.MaxSampleRate != null)
cond.MaxSampleRate = patch.MaxSampleRate.Value;
if (patch.MinBitDepth != null)
cond.MinBitDepth = patch.MinBitDepth.Value;
if (patch.MaxBitDepth != null)
cond.MaxBitDepth = patch.MaxBitDepth.Value;
if (patch.StrictTitle != null)
cond.StrictTitle = patch.StrictTitle.Value;
if (patch.StrictArtist != null)
cond.StrictArtist = patch.StrictArtist.Value;
if (patch.StrictAlbum != null)
cond.StrictAlbum = patch.StrictAlbum.Value;
if (patch.Formats != null)
cond.Formats = patch.Formats;
if (patch.BannedUsers != null)
cond.BannedUsers = patch.BannedUsers;
if (patch.StrictStringDiacrRemove != null)
cond.StrictStringDiacrRemove = patch.StrictStringDiacrRemove.Value;
if (patch.AcceptNoLength != null)
cond.AcceptNoLength = patch.AcceptNoLength.Value;
if (patch.AcceptMissingProps != null)
cond.AcceptMissingProps = patch.AcceptMissingProps.Value;
return cond;
}
public override bool Equals(object obj) public override bool Equals(object obj)
{ {
if (obj is FileConditions other) if (obj is FileConditions other)
@ -244,3 +296,24 @@ public class FileConditions
return "Satisfied"; return "Satisfied";
} }
} }
public class FileConditionsPatch
{
public int? LengthTolerance = null;
public int? MinBitrate = null;
public int? MaxBitrate = null;
public int? MinSampleRate = null;
public int? MaxSampleRate = null;
public int? MinBitDepth = null;
public int? MaxBitDepth = null;
public bool? StrictTitle = null;
public bool? StrictArtist = null;
public bool? StrictAlbum = null;
public string[]? Formats = null;
public string[]? BannedUsers = null;
public bool? StrictStringDiacrRemove = null;
public bool? AcceptNoLength = null;
public bool? AcceptMissingProps = null;
}

78
slsk-batchdl/FileManager.cs
View file

@ -7,16 +7,15 @@ using System.Text.RegularExpressions;
using Data; using Data;
using Enums; using Enums;
using System.ComponentModel;
public class FileManager public class FileManager
{ {
readonly TrackListEntry? tle; readonly TrackListEntry tle;
readonly HashSet<Track> organized = new(); readonly HashSet<Track> organized = new();
string? remoteCommonDir; string? remoteCommonDir;
public FileManager() { }
public FileManager(TrackListEntry tle) public FileManager(TrackListEntry tle)
{ {
this.tle = tle; this.tle = tle;
@ -24,18 +23,24 @@ public class FileManager
public string GetSavePath(string sourceFname) public string GetSavePath(string sourceFname)
{ {
return $"{GetSavePathNoExt(sourceFname)}{Path.GetExtension(sourceFname)}"; return GetSavePathNoExt(sourceFname) + Path.GetExtension(sourceFname);
} }
public string GetSavePathNoExt(string sourceFname) public string GetSavePathNoExt(string sourceFname)
{ {
string parent = Config.parentDir;
string name = Utils.GetFileNameWithoutExtSlsk(sourceFname); string name = Utils.GetFileNameWithoutExtSlsk(sourceFname);
name = name.ReplaceInvalidChars(Config.invalidReplaceStr);
string parent = Config.outputFolder; if (tle.defaultFolderName != null)
if (tle != null && tle.placeInSubdir && remoteCommonDir != null)
{ {
parent = Path.Join(parent, tle.defaultFolderName.ReplaceInvalidChars(Config.invalidReplaceStr, removeSlash: false));
}
if (tle.source.Type == TrackType.Album)
{
if (remoteCommonDir == null)
throw new NullReferenceException("Remote common dir needs to be configured to organize album files");
string dirname = Path.GetFileName(remoteCommonDir); string dirname = Path.GetFileName(remoteCommonDir);
string relpath = Path.GetRelativePath(remoteCommonDir, Utils.NormalizedPath(sourceFname)); string relpath = Path.GetRelativePath(remoteCommonDir, Utils.NormalizedPath(sourceFname));
parent = Path.Join(parent, dirname, Path.GetDirectoryName(relpath)); parent = Path.Join(parent, dirname, Path.GetDirectoryName(relpath));
@ -51,17 +56,12 @@ public class FileManager
public void OrganizeAlbum(List<Track> tracks, List<Track>? additionalImages, bool remainingOnly = true) public void OrganizeAlbum(List<Track> tracks, List<Track>? additionalImages, bool remainingOnly = true)
{ {
if (tle == null)
throw new NullReferenceException("TrackListEntry should not be null.");
string outputFolder = Config.outputFolder;
foreach (var track in tracks.Where(t => !t.IsNotAudio)) foreach (var track in tracks.Where(t => !t.IsNotAudio))
{ {
if (remainingOnly && organized.Contains(track)) if (remainingOnly && organized.Contains(track))
continue; continue;
OrganizeAudio(tle, track, track.FirstDownload); OrganizeAudio(track, track.FirstDownload);
} }
bool onlyAdditionalImages = Config.nameFormat.Length == 0; bool onlyAdditionalImages = Config.nameFormat.Length == 0;
@ -83,20 +83,19 @@ public class FileManager
} }
} }
public void OrganizeAudio(TrackListEntry tle, Track track, Soulseek.File? file) public void OrganizeAudio(Track track, Soulseek.File? file)
{ {
if (track.DownloadPath.Length == 0 || !Utils.IsMusicFile(track.DownloadPath)) if (track.DownloadPath.Length == 0 || !Utils.IsMusicFile(track.DownloadPath))
{
return; return;
}
else if (Config.nameFormat.Length == 0) if (Config.nameFormat.Length == 0)
{ {
organized.Add(track); organized.Add(track);
return; return;
} }
string pathPart = SubstituteValues(Config.nameFormat, track, file); string pathPart = SubstituteValues(Config.nameFormat, track, file);
string newFilePath = Path.Join(Config.outputFolder, pathPart + Path.GetExtension(track.DownloadPath)); string newFilePath = Path.Join(Config.parentDir, pathPart + Path.GetExtension(track.DownloadPath));
try try
{ {
@ -141,15 +140,7 @@ public class FileManager
{ {
Directory.CreateDirectory(Path.GetDirectoryName(newPath)); Directory.CreateDirectory(Path.GetDirectoryName(newPath));
Utils.Move(oldPath, newPath); Utils.Move(oldPath, newPath);
Utils.DeleteAncestorsIfEmpty(Path.GetDirectoryName(oldPath), Config.parentDir);
string x = Utils.NormalizedPath(Path.GetFullPath(Config.outputFolder));
string y = Utils.NormalizedPath(Path.GetDirectoryName(oldPath));
while (x.Length > 0 && y.StartsWith(x + '/') && Utils.FileCountRecursive(y) == 0)
{
Directory.Delete(y, true); // hopefully this is fine
y = Utils.NormalizedPath(Path.GetDirectoryName(y));
}
} }
} }
@ -172,25 +163,33 @@ public class FileManager
inner = inner[1..^1]; inner = inner[1..^1];
var options = inner.Split('|'); var options = inner.Split('|');
string chosenOpt = ""; string? chosenOpt = null;
foreach (var opt in options) foreach (var opt in options)
{ {
string[] parts = Regex.Split(opt, @"\([^\)]*\)"); string[] parts = Regex.Split(opt, @"\([^\)]*\)");
string[] result = parts.Where(part => !string.IsNullOrWhiteSpace(part)).ToArray(); string[] result = parts.Where(part => !string.IsNullOrWhiteSpace(part)).ToArray();
if (result.All(x => GetVarValue(x, file, slfile, track).Length > 0)) if (result.All(x => TryGetVarValue(x, file, slfile, track, out string res) && res.Length > 0))
{ {
chosenOpt = opt; chosenOpt = opt;
break; break;
} }
} }
if (chosenOpt == null)
{
chosenOpt = options[^1];
}
chosenOpt = Regex.Replace(chosenOpt, @"\([^()]*\)|[^()]+", match => chosenOpt = Regex.Replace(chosenOpt, @"\([^()]*\)|[^()]+", match =>
{ {
if (match.Value.StartsWith("(") && match.Value.EndsWith(")")) if (match.Value.StartsWith("(") && match.Value.EndsWith(")"))
return match.Value[1..^1].ReplaceInvalidChars(Config.invalidReplaceStr, removeSlash: false); return match.Value[1..^1].ReplaceInvalidChars(Config.invalidReplaceStr, removeSlash: false);
else else
return GetVarValue(match.Value, file, slfile, track); {
TryGetVarValue(match.Value, file, slfile, track, out string res);
return res;
}
}); });
string old = match.Groups[1].Value; string old = match.Groups[1].Value;
@ -213,10 +212,8 @@ public class FileManager
return format; return format;
} }
string GetVarValue(string x, TagLib.File? file, Soulseek.File? slfile, Track track) bool TryGetVarValue(string x, TagLib.File? file, Soulseek.File? slfile, Track track, out string res)
{ {
string res;
switch (x) switch (x)
{ {
case "artist": case "artist":
@ -249,23 +246,24 @@ public class FileManager
case "foldername": case "foldername":
if (remoteCommonDir == null || slfile == null) if (remoteCommonDir == null || slfile == null)
{ {
return Utils.GetBaseNameSlsk(Utils.GetDirectoryNameSlsk(slfile?.Filename ?? "")); res = Utils.GetBaseNameSlsk(Utils.GetDirectoryNameSlsk(slfile?.Filename ?? ""));
return true;
} }
else else
{ {
string d = Path.GetDirectoryName(Utils.NormalizedPath(slfile.Filename)); string d = Path.GetDirectoryName(Utils.NormalizedPath(slfile.Filename));
string r = Path.GetFileName(remoteCommonDir); string r = Path.GetFileName(remoteCommonDir);
return Path.Join(r, Path.GetRelativePath(remoteCommonDir, d)); res = Path.Join(r, Path.GetRelativePath(remoteCommonDir, d));
return true;
} }
case "default-foldername":
res = Config.defaultFolderName; break;
case "extractor": case "extractor":
res = Config.inputType.ToString(); break; res = Config.inputType.ToString(); break;
default: default:
res = ""; break; res = x; return false;
} }
return res.ReplaceInvalidChars(Config.invalidReplaceStr); res = res.ReplaceInvalidChars(Config.invalidReplaceStr);
return true;
} }
} }

55
slsk-batchdl/Help.cs
View file

@ -19,8 +19,7 @@ public static class Help
General Options General Options
-p, --path <path> Download directory -p, --path <path> Download directory
-f, --folder <name> Subfolder name. Set to '.' to output directly to --path --input-type <type> [csv|youtube|spotify|bandcamp|string|list]
--input-type <type> Force set input type, [csv|youtube|spotify|bandcamp|string]
--name-format <format> Name format for downloaded tracks. See --help name-format --name-format <format> Name format for downloaded tracks. See --help name-format
-n, --number <maxtracks> Download the first n tracks of a playlist -n, --number <maxtracks> Download the first n tracks of a playlist
@ -161,9 +160,6 @@ public static class Help
-t, --interactive Interactive mode, allows to select the folder and images -t, --interactive Interactive mode, allows to select the folder and images
--album-track-count <num> Specify the exact number of tracks in the album. Add a + or --album-track-count <num> Specify the exact number of tracks in the album. Add a + or
- for inequalities, e.g '5+' for five or more tracks. - for inequalities, e.g '5+' for five or more tracks.
--album-ignore-fails Do not skip to the next source and do not delete all
successfully downloaded files if one of the files in the
folder fails to download
--album-art <option> Retrieve additional images after downloading the album: --album-art <option> Retrieve additional images after downloading the album:
'default': No additional images 'default': No additional images
'largest': Download from the folder with the largest image 'largest': Download from the folder with the largest image
@ -171,10 +167,16 @@ public static class Help
--album-art-only Only download album art for the provided album --album-art-only Only download album art for the provided album
--no-browse-folder Do not automatically browse user shares to get all files in --no-browse-folder Do not automatically browse user shares to get all files in
in the folder in the folder
--failed-album-path Path to move all album files to when one of the items from
the directory fails to download. Set to 'delete' to delete
the files instead. Set to the empty string """" to disable.
Default: {configured output dir}/failed
Aggregate Download Aggregate Download
-g, --aggregate Aggregate download mode: Find and download all distinct -g, --aggregate Aggregate download mode: Find and download all distinct
songs associated with the provided artist, album, or title. songs associated with the provided artist, album, or title.
--aggregate-length-tol <tol> Max length tolerance in seconds to consider two tracks or
albums equal. (Default: 3)
--min-shares-aggregate <num> Minimum number of shares of a track or album for it to be --min-shares-aggregate <num> Minimum number of shares of a track or album for it to be
downloaded in aggregate mode. (Default: 2) downloaded in aggregate mode. (Default: 2)
--relax-filtering Slightly relax file filtering in aggregate mode to include --relax-filtering Slightly relax file filtering in aggregate mode to include
@ -196,7 +198,7 @@ public static class Help
Input types Input types
The input type is usually determined automatically. To force a specific input type, set The input type is usually determined automatically. To force a specific input type, set
--input-type [spotify|youtube|csv|string|bandcamp]. The following input types are available: --input-type [spotify|youtube|csv|string|bandcamp|list]. The following input types are available:
CSV file CSV file
Path to a local CSV file: Use a csv file containing track info of the songs to download. Path to a local CSV file: Use a csv file containing track info of the songs to download.
@ -259,6 +261,7 @@ public static class Help
album album
length (in seconds) length (in seconds)
artist-maybe-wrong artist-maybe-wrong
album-track-count
Example inputs and their interpretations: Example inputs and their interpretations:
Input String | Artist | Title | Album | Length Input String | Artist | Title | Album | Length
@ -268,6 +271,17 @@ public static class Help
'Foo - Bar' (with --album enabled) | Foo | | Bar | 'Foo - Bar' (with --album enabled) | Foo | | Bar |
'Artist - Title, length=42' | Artist | Title | | 42 'Artist - Title, length=42' | Artist | Title | | 42
'artist=AR, title=T, album=AL' | AR | T | AL | 'artist=AR, title=T, album=AL' | AR | T | AL |
List
A path to a text file where each line has the following form:
""some input"" ""conditions"" ""preferred conditions""
""album=Album"" ""format=mp3; br > 128"" ""br >= 320""
Where ""some input"" is any of the above input types. The quotes can be omitted if the field
contains no spaces. The conditions and preferred conditions fields are added on top of the
configured conditions and can also be omitted. List input must be manually activated with
--input-type=list.
"; ";
const string downloadModesHelp = @" const string downloadModesHelp = @"
@ -282,24 +296,18 @@ public static class Help
or csv row has no track title, or when -a/--album is enabled. or csv row has no track title, or when -a/--album is enabled.
Aggregate Aggregate
With -g/--aggregate, sldl will first perform an ordinary search for the input, then attempt to With -g/--aggregate, sldl performs an ordinary search for the input then attempts to
group the results into distinct songs and download one of each kind. A common use case is group the results into distinct songs and download one of each kind, starting with the one
finding all remixes of a song or printing all songs by an artist that are not your music dir. which is shared by the most users.
Two files are considered equal if their inferred track title and artist name are equal Note that --min-shares-aggregate is 2 by default, which means that songs shared by only
(ignoring case and some special characters), and their lengths are within --length-tol of each one user will be ignored.
other.
Note that this mode is not 100% reliable, which is why --min-shares-aggregate is set to 2 by
default, i.e. any song that is shared only once will be ignored.
Album Aggregate Album Aggregate
Activated when --album and --aggregate are enabled, in this mode sldl searches for the query Activated when both --album and --aggregate are enabled. sldl will group shares and download
and groups results into distinct albums. Two folders are considered same if they have the one of each distinct album, starting with the one shared by the most users. It's
same number of audio files, and the durations of the files are within --length-tol of each recommended to pair this with --interactive.
other (or within 3 seconds if length-tol is not configured). If both folders have exactly one Note that --min-shares-aggregate is 2 by default, which means that albums shared by only
audio file with similar lengths, also checks if the inferred title and artist name coincide. one user will be ignored.
More reliable than normal aggregate due to much simpler grouping logic.
Note that --min-shares-aggregate is 2 by default, which means that folders shared only once
will be ignored.
"; ";
const string searchHelp = @" const string searchHelp = @"
@ -378,7 +386,7 @@ public static class Help
ranking with this option due to the bitrate and samplerate checks. ranking with this option due to the bitrate and samplerate checks.
Conditions can also be supplied as a semicolon-delimited string with --cond and --pref, e.g Conditions can also be supplied as a semicolon-delimited string with --cond and --pref, e.g
--cond ""br>=320;f=mp3,ogg;sr<96000"" --cond ""br >= 320; format = mp3,ogg; sr < 96000"".
"; ";
const string nameFormatHelp = @" const string nameFormatHelp = @"
@ -414,7 +422,6 @@ public static class Help
disc Disc number disc Disc number
filename Soulseek filename without extension filename Soulseek filename without extension
foldername Soulseek folder name foldername Soulseek folder name
default-foldername Default sldl folder name
extractor Name of the extractor used (CSV/Spotify/YouTube/etc) extractor Name of the extractor used (CSV/Spotify/YouTube/etc)
"; ";

33
slsk-batchdl/M3uEditor.cs
View file

@ -5,25 +5,34 @@ using System.Text;
public class M3uEditor public class M3uEditor
{ {
public string path { get; private set; }
string parent;
List<string> lines; List<string> lines;
bool needFirstUpdate = false; bool needFirstUpdate = false;
readonly TrackLists trackLists; readonly TrackLists trackLists;
readonly string path;
readonly string parent;
readonly int offset = 0;
readonly M3uOption option = M3uOption.Index; readonly M3uOption option = M3uOption.Index;
readonly Dictionary<string, Track> previousRunData = new(); // { track.ToKey(), track } readonly Dictionary<string, Track> previousRunData = new(); // { track.ToKey(), track }
public M3uEditor(string m3uPath, TrackLists trackLists, M3uOption option, int offset = 0) public M3uEditor(TrackLists trackLists, M3uOption option)
{ {
this.trackLists = trackLists; this.trackLists = trackLists;
this.offset = offset;
this.option = option; this.option = option;
this.path = Path.GetFullPath(m3uPath);
this.parent = Utils.NormalizedPath(Path.GetDirectoryName(path));
this.lines = ReadAllLines().ToList();
this.needFirstUpdate = option == M3uOption.All; this.needFirstUpdate = option == M3uOption.All;
}
public M3uEditor(string path, TrackLists trackLists, M3uOption option) : this(trackLists, option)
{
SetPathAndLoad(path);
}
public void SetPathAndLoad(string path)
{
if (this.path == path)
return;
this.path = Path.GetFullPath(path);
parent = Utils.NormalizedPath(Path.GetDirectoryName(this.path));
lines = ReadAllLines().ToList();
LoadPreviousResults(); LoadPreviousResults();
} }
@ -72,7 +81,7 @@ public class M3uEditor
if (field == 0) if (field == 0)
{ {
if (x.StartsWith("./")) if (x.StartsWith("./"))
x = Path.Join(parent, x[2..]); x = System.IO.Path.Join(parent, x[2..]);
track.DownloadPath = x; track.DownloadPath = x;
} }
else if (field == 1) else if (field == 1)
@ -116,7 +125,7 @@ public class M3uEditor
lock (trackLists) lock (trackLists)
{ {
bool needUpdate = false; bool needUpdate = false;
int index = 1 + offset; int index = 1;
bool updateLine(string newLine) bool updateLine(string newLine)
{ {
@ -246,7 +255,7 @@ public class M3uEditor
{ {
string p = val.DownloadPath; string p = val.DownloadPath;
if (Utils.NormalizedPath(p).StartsWith(parent)) if (Utils.NormalizedPath(p).StartsWith(parent))
p = "./" + Path.GetRelativePath(parent, p); // prepend ./ for LoadPreviousResults to recognize that a rel. path is used p = "./" + System.IO.Path.GetRelativePath(parent, p); // prepend ./ for LoadPreviousResults to recognize that a rel. path is used
var items = new string[] var items = new string[]
{ {
@ -278,7 +287,7 @@ public class M3uEditor
if (track.DownloadPath.Length > 0) if (track.DownloadPath.Length > 0)
{ {
if (track.DownloadPath.StartsWith(parent)) if (Utils.NormalizedPath(track.DownloadPath).StartsWith(parent))
return Path.GetRelativePath(parent, track.DownloadPath); return Path.GetRelativePath(parent, track.DownloadPath);
else else
return track.DownloadPath; return track.DownloadPath;

283
slsk-batchdl/Program.cs
View file

@ -5,10 +5,12 @@ using System.Collections.Concurrent;
using System.Data; using System.Data;
using System.Diagnostics; using System.Diagnostics;
using System.Text.RegularExpressions; using System.Text.RegularExpressions;
using System.Net.Sockets;
using Data; using Data;
using Enums; using Enums;
using FileSkippers; using FileSkippers;
using Extractors;
using static Printing; using static Printing;
using Directory = System.IO.Directory; using Directory = System.IO.Directory;
@ -54,14 +56,11 @@ static partial class Program
if (Config.input.Length == 0) if (Config.input.Length == 0)
throw new ArgumentException($"No input provided"); throw new ArgumentException($"No input provided");
(Config.inputType, extractor) = Extractors.ExtractorRegistry.GetMatchingExtractor(Config.input); (Config.inputType, extractor) = ExtractorRegistry.GetMatchingExtractor(Config.input, Config.inputType);
if (Config.inputType == InputType.None)
throw new ArgumentException($"No matching extractor for input '{Config.input}'");
WriteLine($"Using extractor: {Config.inputType}", debugOnly: true); WriteLine($"Using extractor: {Config.inputType}", debugOnly: true);
trackLists = await extractor.GetTracks(Config.maxTracks, Config.offset, Config.reverse); trackLists = await extractor.GetTracks(Config.input, Config.maxTracks, Config.offset, Config.reverse);
WriteLine("Got tracks", debugOnly: true); WriteLine("Got tracks", debugOnly: true);
@ -71,7 +70,7 @@ static partial class Program
Config.PostProcessArgs(); Config.PostProcessArgs();
m3uEditor = new M3uEditor(Config.m3uFilePath, trackLists, Config.m3uOption, Config.offset); m3uEditor = new M3uEditor(trackLists, Config.m3uOption);
InitFileSkippers(); InitFileSkippers();
@ -88,9 +87,25 @@ static partial class Program
bool needLogin = !Config.PrintTracks; bool needLogin = !Config.PrintTracks;
if (needLogin) if (needLogin)
{ {
client = new SoulseekClient(new SoulseekClientOptions(listenPort: Config.listenPort)); var connectionOptions = new ConnectionOptions(configureSocket: (socket) =>
{
socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.KeepAlive, true);
socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.TcpKeepAliveRetryCount, 3);
socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.TcpKeepAliveTime, 15);
socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.TcpKeepAliveInterval, 15);
});
var clientOptions = new SoulseekClientOptions(
transferConnectionOptions: connectionOptions,
serverConnectionOptions: connectionOptions,
listenPort: Config.listenPort
);
client = new SoulseekClient(clientOptions);
if (!Config.useRandomLogin && (string.IsNullOrEmpty(Config.username) || string.IsNullOrEmpty(Config.password))) if (!Config.useRandomLogin && (string.IsNullOrEmpty(Config.username) || string.IsNullOrEmpty(Config.password)))
throw new ArgumentException("No soulseek username or password"); throw new ArgumentException("No soulseek username or password");
await Login(Config.useRandomLogin); await Login(Config.useRandomLogin);
Search.searchSemaphore = new RateLimitedSemaphore(Config.searchesPerTime, TimeSpan.FromSeconds(Config.searchRenewTime)); Search.searchSemaphore = new RateLimitedSemaphore(Config.searchesPerTime, TimeSpan.FromSeconds(Config.searchRenewTime));
@ -113,8 +128,8 @@ static partial class Program
{ {
var cond = Config.skipExistingPrefCond ? Config.preferredCond : Config.necessaryCond; var cond = Config.skipExistingPrefCond ? Config.preferredCond : Config.necessaryCond;
if (Config.musicDir.Length == 0 || !Config.outputFolder.StartsWith(Config.musicDir, StringComparison.OrdinalIgnoreCase)) if (Config.musicDir.Length == 0 || !Config.parentDir.StartsWith(Config.musicDir, StringComparison.OrdinalIgnoreCase))
outputDirSkipper = FileSkipperRegistry.GetChecker(Config.skipMode, Config.outputFolder, cond, m3uEditor); outputDirSkipper = FileSkipperRegistry.GetChecker(Config.skipMode, Config.parentDir, cond, m3uEditor);
if (Config.musicDir.Length > 0) if (Config.musicDir.Length > 0)
{ {
@ -171,6 +186,27 @@ static partial class Program
} }
static void PrepareListEntry(TrackListEntry tle)
{
Config.RestoreConditions();
Config.UpdateProfiles(tle);
Config.AddTemporaryConditions(tle.additionalConds, tle.additionalPrefConds);
string m3uPath;
if (Config.m3uFilePath.Length > 0)
m3uPath = Config.m3uFilePath;
else
m3uPath = Path.Join(Config.parentDir, tle.defaultFolderName, "sldl.m3u");
m3uEditor.SetPathAndLoad(m3uPath);
PreprocessTracks(tle);
}
static async Task MainLoop() static async Task MainLoop()
{ {
for (int i = 0; i < trackLists.lists.Count; i++) for (int i = 0; i < trackLists.lists.Count; i++)
@ -179,13 +215,10 @@ static partial class Program
var tle = trackLists[i]; var tle = trackLists[i];
Config.UpdateArgs(tle); PrepareListEntry(tle);
PreprocessTracks(tle);
var existing = new List<Track>(); var existing = new List<Track>();
var notFound = new List<Track>(); var notFound = new List<Track>();
var responseData = new ResponseData();
if (Config.skipNotFound && !Config.PrintResults) if (Config.skipNotFound && !Config.PrintResults)
{ {
@ -247,13 +280,18 @@ static partial class Program
Console.WriteLine($"{tle.source.Type} download: {tle.source.ToString(true)}, searching.."); Console.WriteLine($"{tle.source.Type} download: {tle.source.ToString(true)}, searching..");
bool foundSomething = false;
var responseData = new ResponseData();
if (tle.source.Type == TrackType.Album) if (tle.source.Type == TrackType.Album)
{ {
tle.list = await Search.GetAlbumDownloads(tle.source, responseData); tle.list = await Search.GetAlbumDownloads(tle.source, responseData);
foundSomething = tle.list.Count > 0;
} }
else if (tle.source.Type == TrackType.Aggregate) else if (tle.source.Type == TrackType.Aggregate)
{ {
tle.list.Insert(0, await Search.GetAggregateTracks(tle.source, responseData)); tle.list.Insert(0, await Search.GetAggregateTracks(tle.source, responseData));
foundSomething = tle.list.Count > 0;
} }
else if (tle.source.Type == TrackType.AlbumAggregate) else if (tle.source.Type == TrackType.AlbumAggregate)
{ {
@ -262,8 +300,27 @@ static partial class Program
foreach (var item in res) foreach (var item in res)
{ {
var newSource = new Track(tle.source) { Type = TrackType.Album }; var newSource = new Track(tle.source) { Type = TrackType.Album };
trackLists.AddEntry(new TrackListEntry(item, newSource, false, true, true, false, false)); var albumTle = new TrackListEntry(item, newSource, needSourceSearch: false, sourceCanBeSkipped: true);
albumTle.defaultFolderName = tle.defaultFolderName;
trackLists.AddEntry(albumTle);
} }
foundSomething = res.Count > 0;
}
if (!foundSomething)
{
var lockedFiles = responseData.lockedFilesCount > 0 ? $" (Found {responseData.lockedFilesCount} locked files)" : "";
Console.WriteLine($"No results.{lockedFiles}");
if (!Config.PrintResults)
{
tle.source.State = TrackState.Failed;
tle.source.FailureReason = FailureReason.NoSuitableFileFound;
m3uEditor.Update();
}
continue;
} }
if (Config.skipExisting && tle.needSkipExistingAfterSearch) if (Config.skipExisting && tle.needSkipExistingAfterSearch)
@ -284,18 +341,6 @@ static partial class Program
continue; continue;
} }
if (tle.needSourceSearch && (tle.list.Count == 0 || !tle.list.Any(x => x.Count > 0)))
{
string lockedFilesStr = responseData.lockedFilesCount > 0 ? $" (Found {responseData.lockedFilesCount} locked files)" : "";
Console.WriteLine($"No results.{lockedFilesStr}");
tle.source.State = TrackState.Failed;
tle.source.FailureReason = FailureReason.NoSuitableFileFound;
m3uEditor.Update();
continue;
}
m3uEditor.Update(); m3uEditor.Update();
if (tle.source.Type != TrackType.Album) if (tle.source.Type != TrackType.Album)
@ -416,11 +461,15 @@ static partial class Program
var downloadTasks = tracks.Select(async (track, index) => var downloadTasks = tracks.Select(async (track, index) =>
{ {
await DownloadTask(tle, track, semaphore, organizer, null, false, true); using var cts = new CancellationTokenSource();
await DownloadTask(tle, track, semaphore, organizer, cts, false, true, true);
m3uEditor.Update(); m3uEditor.Update();
}); });
await Task.WhenAll(downloadTasks); await Task.WhenAll(downloadTasks);
if (Config.removeTracksFromSource && tracks.All(t => t.State == TrackState.Downloaded || t.State == TrackState.AlreadyExists))
await extractor.RemoveTrackFromSource(tle.source);
} }
@ -449,13 +498,6 @@ static partial class Program
organizer.SetRemoteCommonDir(soulseekDir); organizer.SetRemoteCommonDir(soulseekDir);
if (!Config.noBrowseFolder && !Config.interactiveMode && !retrievedFolders.Contains(soulseekDir))
{
Console.WriteLine("Getting all files in folder...");
await Search.CompleteFolder(tracks, tracks[0].FirstResponse, soulseekDir);
retrievedFolders.Add(tracks[0].FirstUsername + '\\' + soulseekDir);
}
if (!Config.interactiveMode && !wasInteractive) if (!Config.interactiveMode && !wasInteractive)
{ {
Console.WriteLine(); Console.WriteLine();
@ -463,36 +505,80 @@ static partial class Program
} }
var semaphore = new SemaphoreSlim(Config.concurrentProcesses); var semaphore = new SemaphoreSlim(Config.concurrentProcesses);
var cts = new CancellationTokenSource(); using var cts = new CancellationTokenSource();
try try
{ {
var downloadTasks = tracks.Select(async track => await RunAlbumDownloads(tle, organizer, tracks, semaphore, cts);
if (!Config.noBrowseFolder && !retrievedFolders.Contains(soulseekDir))
{ {
await DownloadTask(tle, track, semaphore, organizer, cts, cancelOnFail: !Config.albumIgnoreFails, true); Console.WriteLine("Getting all files in folder...");
});
await Task.WhenAll(downloadTasks); int newFilesFound = await Search.CompleteFolder(tracks, tracks[0].FirstResponse, soulseekDir);
retrievedFolders.Add(tracks[0].FirstUsername + '\\' + soulseekDir);
if (newFilesFound > 0)
{
Console.WriteLine($"Found {newFilesFound} more files in the directory, downloading:");
await RunAlbumDownloads(tle, organizer, tracks, semaphore, cts);
}
else
{
Console.WriteLine("No more files found.");
}
}
succeeded = true; succeeded = true;
break; break;
} }
catch (OperationCanceledException) when (!Config.albumIgnoreFails) catch (OperationCanceledException)
{ {
foreach (var track in tracks) OnAlbumFail(tracks);
{
if (track.State == TrackState.Downloaded && File.Exists(track.DownloadPath))
{
try { File.Delete(track.DownloadPath); } catch { }
}
}
} }
organizer.SetRemoteCommonDir(null); organizer.SetRemoteCommonDir(null);
tle.list.RemoveAt(index); tle.list.RemoveAt(index);
} }
if (tracks != null && succeeded) if (succeeded)
{ {
await OnAlbumSuccess(tle, tracks);
}
List<Track>? additionalImages = null;
if (Config.albumArtOnly || succeeded && Config.albumArtOption != AlbumArtOption.Default)
{
Console.WriteLine($"\nDownloading additional images:");
additionalImages = await DownloadImages(tle.list, Config.albumArtOption, tracks, organizer);
tracks?.AddRange(additionalImages);
}
if (tracks != null && tle.source.DownloadPath.Length > 0)
{
organizer.OrganizeAlbum(tracks, additionalImages);
}
m3uEditor.Update();
}
static async Task RunAlbumDownloads(TrackListEntry tle, FileManager organizer, List<Track> tracks, SemaphoreSlim semaphore, CancellationTokenSource cts)
{
var downloadTasks = tracks.Select(async track =>
{
await DownloadTask(tle, track, semaphore, organizer, cts, true, true, true);
});
await Task.WhenAll(downloadTasks);
}
static async Task OnAlbumSuccess(TrackListEntry tle, List<Track>? tracks)
{
if (tracks == null)
return;
var downloadedAudio = tracks.Where(t => !t.IsNotAudio && t.State == TrackState.Downloaded && t.DownloadPath.Length > 0); var downloadedAudio = tracks.Where(t => !t.IsNotAudio && t.State == TrackState.Downloaded && t.DownloadPath.Length > 0);
if (downloadedAudio.Any()) if (downloadedAudio.Any())
@ -507,25 +593,41 @@ static partial class Program
} }
} }
List<Track>? additionalImages = null;
if (Config.albumArtOnly || succeeded && Config.albumArtOption != AlbumArtOption.Default) static void OnAlbumFail(List<Track>? tracks)
{ {
Console.WriteLine($"\nDownloading additional images:"); if (tracks == null || Config.IgnoreAlbumFail)
additionalImages = await DownloadImages(tle.list, Config.albumArtOption, tracks); return;
tracks?.AddRange(additionalImages);
}
if (tracks != null && tle.source.DownloadPath.Length > 0) foreach (var track in tracks)
{ {
organizer.OrganizeAlbum(tracks, additionalImages); if (track.DownloadPath.Length > 0 && File.Exists(track.DownloadPath))
{
try
{
if (Config.DeleteAlbumOnFail)
{
File.Delete(track.DownloadPath);
}
else if (Config.failedAlbumPath.Length > 0)
{
var newPath = Path.Join(Config.failedAlbumPath, Path.GetRelativePath(Config.parentDir, track.DownloadPath));
Directory.CreateDirectory(Path.GetDirectoryName(newPath));
Utils.Move(track.DownloadPath, newPath);
} }
m3uEditor.Update(); Utils.DeleteAncestorsIfEmpty(Path.GetDirectoryName(track.DownloadPath), Config.parentDir);
}
catch (Exception e)
{
Printing.WriteLine($"Error: Unable to move or delete file '{track.DownloadPath}' after album fail: {e}");
}
}
}
} }
static async Task<List<Track>> DownloadImages(List<List<Track>> downloads, AlbumArtOption option, List<Track>? chosenAlbum) static async Task<List<Track>> DownloadImages(List<List<Track>> downloads, AlbumArtOption option, List<Track>? chosenAlbum, FileManager fileManager)
{ {
var downloadedImages = new List<Track>(); var downloadedImages = new List<Track>();
long mSize = 0; long mSize = 0;
@ -621,11 +723,11 @@ static partial class Program
bool allSucceeded = true; bool allSucceeded = true;
var semaphore = new SemaphoreSlim(1); var semaphore = new SemaphoreSlim(1);
var organizer = new FileManager();
foreach (var track in tracks) foreach (var track in tracks)
{ {
await DownloadTask(null, track, semaphore, organizer, null, false, false); using var cts = new CancellationTokenSource();
await DownloadTask(null, track, semaphore, fileManager, cts, false, false, false);
if (track.State == TrackState.Downloaded) if (track.State == TrackState.Downloaded)
downloadedImages.Add(track); downloadedImages.Add(track);
@ -641,15 +743,12 @@ static partial class Program
} }
static async Task DownloadTask(TrackListEntry? tle, Track track, SemaphoreSlim semaphore, FileManager organizer, CancellationTokenSource? cts, bool cancelOnFail, bool removeFromSource) static async Task DownloadTask(TrackListEntry? tle, Track track, SemaphoreSlim semaphore, FileManager organizer, CancellationTokenSource cts, bool cancelOnFail, bool removeFromSource, bool organize)
{ {
if (track.State != TrackState.Initial) if (track.State != TrackState.Initial)
return; return;
if (cts != null)
await semaphore.WaitAsync(cts.Token); await semaphore.WaitAsync(cts.Token);
else
await semaphore.WaitAsync();
int tries = Config.unknownErrorRetries; int tries = Config.unknownErrorRetries;
string savedFilePath = ""; string savedFilePath = "";
@ -659,15 +758,15 @@ static partial class Program
{ {
await WaitForLogin(); await WaitForLogin();
cts?.Token.ThrowIfCancellationRequested(); cts.Token.ThrowIfCancellationRequested();
try try
{ {
(savedFilePath, chosenFile) = await Search.SearchAndDownload(track, organizer); (savedFilePath, chosenFile) = await Search.SearchAndDownload(track, organizer, cts);
} }
catch (Exception ex) catch (Exception ex)
{ {
WriteLine($"Exception thrown: {ex}", debugOnly: true); WriteLine($"Error: {ex}", debugOnly: true);
if (!IsConnectedAndLoggedIn()) if (!IsConnectedAndLoggedIn())
{ {
continue; continue;
@ -682,13 +781,12 @@ static partial class Program
if (cancelOnFail) if (cancelOnFail)
{ {
cts?.Cancel(); cts.Cancel();
throw new OperationCanceledException(); throw new OperationCanceledException();
} }
} }
else else
{ {
WriteLine($"\n{ex.Message}\n{ex.StackTrace}\n", ConsoleColor.DarkYellow, true);
tries--; tries--;
continue; continue;
} }
@ -699,7 +797,7 @@ static partial class Program
if (tries == 0 && cancelOnFail) if (tries == 0 && cancelOnFail)
{ {
cts?.Cancel(); cts.Cancel();
throw new OperationCanceledException(); throw new OperationCanceledException();
} }
@ -724,9 +822,9 @@ static partial class Program
} }
} }
if (track.State == TrackState.Downloaded && tle != null) if (track.State == TrackState.Downloaded && organize)
{ {
organizer?.OrganizeAudio(tle, track, chosenFile); organizer?.OrganizeAudio(track, chosenFile);
} }
if (Config.onComplete.Length > 0) if (Config.onComplete.Length > 0)
@ -760,7 +858,13 @@ static partial class Program
} }
} }
WriteLine($"\nPrev [Up/p] / Next [Down/n] / Accept [Enter] / Accept & Exit Interactive [q] / Skip [Esc/s]\n", ConsoleColor.Green); string retrieveAll1 = retrieveFolder ? "| [r] " : "";
string retrieveAll2 = retrieveFolder ? "| Load All Files " : "";
Console.WriteLine();
WriteLine($" [Up/p] | [Down/n] | [Enter] | [q] {retrieveAll1}| [Esc/s]", ConsoleColor.Green);
WriteLine($" Prev | Next | Accept | Accept & Quit Interactive {retrieveAll2}| Skip", ConsoleColor.Green);
Console.WriteLine();
while (true) while (true)
{ {
@ -770,17 +874,10 @@ static partial class Program
WriteLine($"[{aidx + 1} / {list.Count}]", ConsoleColor.DarkGray); WriteLine($"[{aidx + 1} / {list.Count}]", ConsoleColor.DarkGray);
var folder = Utils.GreatestCommonDirectorySlsk(tracks.Select(t => t.FirstDownload.Filename));
if (retrieveFolder && !retrievedFolders.Contains(username + '\\' + folder))
{
Console.WriteLine("Getting all files in folder...");
await Search.CompleteFolder(tracks, response, folder);
retrievedFolders.Add(username + '\\' + folder);
}
PrintAlbum(tracks); PrintAlbum(tracks);
Console.WriteLine(); Console.WriteLine();
Loop:
string userInput = interactiveModeLoop().Trim(); string userInput = interactiveModeLoop().Trim();
switch (userInput) switch (userInput)
{ {
@ -795,6 +892,24 @@ static partial class Program
case "q": case "q":
Config.interactiveMode = false; Config.interactiveMode = false;
return aidx; return aidx;
case "r":
var folder = Utils.GreatestCommonDirectorySlsk(tracks.Select(t => t.FirstDownload.Filename));
if (retrieveFolder && !retrievedFolders.Contains(username + '\\' + folder))
{
Console.WriteLine("Getting all files in folder...");
int newFiles = await Search.CompleteFolder(tracks, response, folder);
retrievedFolders.Add(username + '\\' + folder);
if (newFiles == 0)
{
Console.WriteLine("No more files found.");
goto Loop;
}
else
{
Console.WriteLine($"Found {newFiles} more files in the folder:");
}
}
break;
case "": case "":
return aidx; return aidx;
} }
@ -846,7 +961,9 @@ static partial class Program
} }
else else
{ {
if (!client.State.HasFlag(SoulseekClientStates.LoggedIn | SoulseekClientStates.LoggingIn | SoulseekClientStates.Connecting)) if (!client.State.HasFlag(SoulseekClientStates.LoggedIn)
&& !client.State.HasFlag(SoulseekClientStates.LoggingIn)
&& !client.State.HasFlag(SoulseekClientStates.Connecting))
{ {
WriteLine($"\nDisconnected, logging in\n", ConsoleColor.DarkYellow, true); WriteLine($"\nDisconnected, logging in\n", ConsoleColor.DarkYellow, true);
try { await Login(Config.useRandomLogin); } try { await Login(Config.useRandomLogin); }
@ -1017,7 +1134,7 @@ static partial class Program
public static bool IsConnectedAndLoggedIn() public static bool IsConnectedAndLoggedIn()
{ {
return client != null && (client.State & (SoulseekClientStates.Connected | SoulseekClientStates.LoggedIn)) != 0; return client != null && client.State.HasFlag(SoulseekClientStates.Connected) && client.State.HasFlag(SoulseekClientStates.LoggedIn);
} }
} }

50
slsk-batchdl/Search.cs
View file

@ -20,17 +20,17 @@ static class Search
public static RateLimitedSemaphore? searchSemaphore; public static RateLimitedSemaphore? searchSemaphore;
// very messy function that does everything // very messy function that does everything
public static async Task<(string, SlFile?)> SearchAndDownload(Track track, FileManager organizer) public static async Task<(string, SlFile?)> SearchAndDownload(Track track, FileManager organizer, CancellationTokenSource cts)
{ {
if (Config.DoNotDownload) if (Config.DoNotDownload)
throw new Exception(); throw new Exception();
var responseData = new ResponseData();
IEnumerable<(SlResponse response, SlFile file)>? orderedResults = null; IEnumerable<(SlResponse response, SlFile file)>? orderedResults = null;
var responseData = new ResponseData();
var progress = Printing.GetProgressBar(Config.displayMode); var progress = Printing.GetProgressBar(Config.displayMode);
var results = new SlDictionary(); var results = new SlDictionary();
var fsResults = new SlDictionary(); var fsResults = new SlDictionary();
var cts = new CancellationTokenSource(); using var searchCts = new CancellationTokenSource();
var saveFilePath = ""; var saveFilePath = "";
SlFile? chosenFile = null; SlFile? chosenFile = null;
Task? downloadTask = null; Task? downloadTask = null;
@ -62,7 +62,7 @@ static class Search
saveFilePath = organizer.GetSavePath(f.Filename); saveFilePath = organizer.GetSavePath(f.Filename);
fsUser = r.Username; fsUser = r.Username;
chosenFile = f; chosenFile = f;
downloadTask = Download.DownloadFile(r, f, saveFilePath, track, progress, cts); downloadTask = Download.DownloadFile(r, f, saveFilePath, track, progress, cts, searchCts);
} }
} }
} }
@ -111,7 +111,7 @@ static class Search
} }
void onSearch() => Printing.RefreshOrPrint(progress, 0, $"Searching: {track}", true); void onSearch() => Printing.RefreshOrPrint(progress, 0, $"Searching: {track}", true);
await RunSearches(track, results, getSearchOptions, responseHandler, cts.Token, onSearch); await RunSearches(track, results, getSearchOptions, responseHandler, searchCts.Token, onSearch);
searches.TryRemove(track, out _); searches.TryRemove(track, out _);
searchEnded = true; searchEnded = true;
@ -142,7 +142,7 @@ static class Search
} }
} }
cts.Dispose(); searchCts.Dispose();
downloads: downloads:
@ -159,7 +159,7 @@ static class Search
try try
{ {
downloading = 1; downloading = 1;
await Download.DownloadFile(response, file, saveFilePath, track, progress); await Download.DownloadFile(response, file, saveFilePath, track, progress, cts);
userSuccessCount.AddOrUpdate(response.Username, 1, (k, v) => v + 1); userSuccessCount.AddOrUpdate(response.Username, 1, (k, v) => v + 1);
return true; return true;
} }
@ -168,13 +168,17 @@ static class Search
chosenFile = null; chosenFile = null;
saveFilePath = ""; saveFilePath = "";
downloading = 0; downloading = 0;
if (!IsConnectedAndLoggedIn()) if (!IsConnectedAndLoggedIn())
throw; throw;
Printing.WriteLine("Error: " + e.Message, ConsoleColor.DarkYellow, true);
userSuccessCount.AddOrUpdate(response.Username, -1, (k, v) => v - 1); userSuccessCount.AddOrUpdate(response.Username, -1, (k, v) => v - 1);
if (--trackTries <= 0) if (--trackTries <= 0)
{ {
Printing.RefreshOrPrint(progress, 0, $"Out of download retries: {track}", true); Printing.RefreshOrPrint(progress, 0, $"Out of download retries: {track}", true);
Printing.WriteLine("Last error was: " + e.Message, ConsoleColor.DarkYellow, true); Printing.WriteLine("Last error was: " + e.Message, ConsoleColor.DarkYellow);
throw new SearchAndDownloadException(FailureReason.OutOfDownloadRetries); throw new SearchAndDownloadException(FailureReason.OutOfDownloadRetries);
} }
return false; return false;
@ -284,7 +288,7 @@ static class Search
results.TryAdd(r.Username + "\\" + file.Filename, (r, file)); results.TryAdd(r.Username + "\\" + file.Filename, (r, file));
} }
} }
var cts = new CancellationTokenSource(); using var cts = new CancellationTokenSource();
await RunSearches(track, results, getSearchOptions, handler, cts.Token); await RunSearches(track, results, getSearchOptions, handler, cts.Token);
@ -427,7 +431,7 @@ static class Search
results.TryAdd(r.Username + "\\" + file.Filename, (r, file)); results.TryAdd(r.Username + "\\" + file.Filename, (r, file));
} }
} }
var cts = new CancellationTokenSource(); using var cts = new CancellationTokenSource();
await RunSearches(track, results, getSearchOptions, handler, cts.Token); await RunSearches(track, results, getSearchOptions, handler, cts.Token);
@ -461,10 +465,7 @@ static class Search
public static async Task<List<List<List<Track>>>> GetAggregateAlbums(Track track, ResponseData responseData) public static async Task<List<List<List<Track>>>> GetAggregateAlbums(Track track, ResponseData responseData)
{ {
int maxDiff = Config.necessaryCond.LengthTolerance; int maxDiff = Config.aggregateLengthTol;
if (maxDiff < 0)
maxDiff = 3;
bool lengthsAreSimilar(int[] sorted1, int[] sorted2) bool lengthsAreSimilar(int[] sorted1, int[] sorted2)
{ {
@ -555,13 +556,13 @@ static class Search
} }
public static async Task<List<(string dir, SlFile file)>> GetAllFilesInFolder(string user, string folderPrefix) public static async Task<List<(string dir, SlFile file)>> GetAllFilesInFolder(string user, string folderPrefix, CancellationToken? cancellationToken = null)
{ {
var browseOptions = new BrowseOptions(); var browseOptions = new BrowseOptions();
var res = new List<(string dir, SlFile file)>(); var res = new List<(string dir, SlFile file)>();
folderPrefix = folderPrefix.TrimEnd('\\') + '\\'; folderPrefix = folderPrefix.TrimEnd('\\') + '\\';
var userFileList = await client.BrowseAsync(user, browseOptions); var userFileList = await client.BrowseAsync(user, browseOptions, cancellationToken);
foreach (var dir in userFileList.Directories) foreach (var dir in userFileList.Directories)
{ {
@ -572,22 +573,15 @@ static class Search
} }
} }
return res; return res;
// It would be much better to use GetDirectoryContentsAsync. Unfortunately it only returns the file
// names without full paths, and DownloadAsync needs full paths in order to download files.
// Therefore it would not be possible to download any files that are in a subdirectory of the folder.
// var dir = await client.GetDirectoryContentsAsync(user, folderPrefix);
// var res = dir.Files.Select(x => (folderPrefix, x)).ToList();
// return res;
} }
public static async Task CompleteFolder(List<Track> tracks, SearchResponse response, string folder) public static async Task<int> CompleteFolder(List<Track> tracks, SearchResponse response, string folder, CancellationToken? cancellationToken = null)
{ {
int newFiles = 0;
try try
{ {
var allFiles = await GetAllFilesInFolder(response.Username, folder); var allFiles = await GetAllFilesInFolder(response.Username, folder, cancellationToken);
if (allFiles.Count > tracks.Count) if (allFiles.Count > tracks.Count)
{ {
@ -599,6 +593,7 @@ static class Search
var fullPath = dir + '\\' + file.Filename; var fullPath = dir + '\\' + file.Filename;
if (!paths.Contains(fullPath)) if (!paths.Contains(fullPath))
{ {
newFiles++;
var newFile = new SlFile(file.Code, fullPath, file.Size, file.Extension, file.Attributes); var newFile = new SlFile(file.Code, fullPath, file.Size, file.Extension, file.Attributes);
var t = new Track var t = new Track
{ {
@ -616,6 +611,7 @@ static class Search
{ {
Printing.WriteLine($"Error getting complete list of files: {ex}", ConsoleColor.DarkYellow); Printing.WriteLine($"Error getting complete list of files: {ex}", ConsoleColor.DarkYellow);
} }
return newFiles;
} }
@ -633,7 +629,7 @@ static class Search
} }
var groups = fileResponses var groups = fileResponses
.GroupBy(inferTrack, new TrackComparer(ignoreCase: true, Config.necessaryCond.LengthTolerance)) .GroupBy(inferTrack, new TrackComparer(ignoreCase: true, Config.aggregateLengthTol))
.Select(x => (x, x.Select(y => y.Item1.Username).Distinct().Count())) .Select(x => (x, x.Select(y => y.Item1.Username).Distinct().Count()))
.Where(x => x.Item2 >= minShares) .Where(x => x.Item2 >= minShares)
.OrderByDescending(x => x.Item2) .OrderByDescending(x => x.Item2)

6
slsk-batchdl/Test.cs
View file

@ -255,7 +255,7 @@ namespace Test
{ {
Config.input = strings[i]; Config.input = strings[i];
Console.WriteLine(Config.input); Console.WriteLine(Config.input);
var res = await extractor.GetTracks(0, 0, false); var res = await extractor.GetTracks(Config.input, 0, 0, false);
var t = res[0].list[0][0]; var t = res[0].list[0][0];
Assert(Extractors.StringExtractor.InputMatches(Config.input)); Assert(Extractors.StringExtractor.InputMatches(Config.input));
Assert(t.ToKey() == tracks[i].ToKey()); Assert(t.ToKey() == tracks[i].ToKey());
@ -268,7 +268,7 @@ namespace Test
{ {
Config.input = strings[i]; Config.input = strings[i];
Console.WriteLine(Config.input); Console.WriteLine(Config.input);
var t = (await extractor.GetTracks(0, 0, false))[0].source; var t = (await extractor.GetTracks(Config.input, 0, 0, false))[0].source;
Assert(Extractors.StringExtractor.InputMatches(Config.input)); Assert(Extractors.StringExtractor.InputMatches(Config.input));
Assert(t.ToKey() == albums[i].ToKey()); Assert(t.ToKey() == albums[i].ToKey());
} }
@ -317,7 +317,7 @@ namespace Test
}; };
var trackLists = new TrackLists(); var trackLists = new TrackLists();
trackLists.AddEntry(new TrackListEntry()); trackLists.AddEntry(new TrackListEntry(TrackType.Normal));
foreach (var t in notFoundInitial) foreach (var t in notFoundInitial)
trackLists.AddTrackToLast(t); trackLists.AddTrackToLast(t);
foreach (var t in existingInitial) foreach (var t in existingInitial)

27
slsk-batchdl/Utils.cs
View file

@ -89,10 +89,13 @@ public static class Utils
return path; return path;
} }
if (path.StartsWith("~")) if (path.StartsWith('~'))
{ {
string homeDirectory = Environment.GetFolderPath(Environment.SpecialFolder.UserProfile); string homeDirectory = Environment.GetFolderPath(Environment.SpecialFolder.UserProfile);
return Path.Combine(homeDirectory, path.Substring(1).TrimStart('/').TrimStart('\\')); path = Path.Join(homeDirectory, path.Substring(1).TrimStart('/').TrimStart('\\'));
if (path.Length > 0)
path = Path.GetFullPath(path);
} }
return path; return path;
@ -162,6 +165,26 @@ public static class Utils
} }
} }
public static void DeleteAncestorsIfEmpty(string startDir, string root)
{
string x = NormalizedPath(Path.GetFullPath(root));
string y = NormalizedPath(startDir);
if (x.Length == 0)
return;
while (y.StartsWith(x + '/') && FileCountRecursive(y) == 0)
{
Directory.Delete(y, true);
string prev = y;
y = NormalizedPath(Path.GetDirectoryName(y) ?? "");
if (prev.Length == y.Length)
break;
}
}
public static bool EqualsAny(this string input, string[] values, StringComparison comparison = StringComparison.Ordinal) public static bool EqualsAny(this string input, string[] values, StringComparison comparison = StringComparison.Ordinal)
{ {
foreach (var value in values) foreach (var value in values)