Experience has proved programmatically setting joins is not useful, and
getting the types and values of query parts was not being maintained.
The programmatic setting of GROUP BY may be useful in future, however.
In theory the import (as opposed to parse) routine could be used for any
format; this could be used to implement an ad hoc JSON format to avoid
the loss of commas in tags with OPML
Folder 0 (the root folder) is a valid, though nonsensical selection:
using it as a positive option is the same as not using the option at
all, and using it as a negative option necessarily yields an empty set.
However, it can in some contexts be validly specified, and so it should
be handled consistently. It had not been previously, but is now.
- Ensure the last refresh time is included in authenticated requests
- Use a partial mock in auth tests so that other processing does not
get in the way of results
- Make sure the group list includes unused groups
- Make sure the update time of subscriptions is correct
Tokens are similar to sessions in that they stand in for users, but the
protocol handlers will manage them; Fever login hashes are the
originating use case for them. These must never expire, for example,
and we need to specify their values.
This commit also performs a bit of database clean-up
- Exec and lock timeouts now apply to MySQL
- Lock timeout now applies to PostgreSQL
- SQLite now uses a generic lock timeout setting which applies to all
- Each parameter is checked for type and normalized
- Interval strings are converted to DateInterval objects
- Timeouts can be specified as interval strings
- Most intervals can be null to signify infinity
- Driver classes are checked that they implement the correct interface
- Short driver names may be used, and are used by default
- Helpful errors messages are printed in case of erroneous configuration
Exporting is currently broken; this will be fixed in an upcoming commit
Three test failures remain, but these are minor and will be resolved
soon. Handling of binary data is also broken, but given that this works
fine with the PDO driver, there is presumably some correct method.
No testing has been performed yet, but changes are extensive enough to
warrant a commit. Of particular note:
- SQL states are enumerated in a separate trait to reduce duplication
- PDOStatement is now an abstract class to avoid duplication of
engine-specific error handling
- Error handling has been cleaned up somewhat
Reasons for failures included an unhandled error code, erroneous sorting
assumptions, and a broken computation of the next insert ID in tests
Five failures remain.
This involved changes to the driver interface as well as the database
schemata. The most significantly altered queries were for article
selection and marking, which relied upon unusual features of SQLite.
Overall query efficiency should not be adversely affected (it may have
even imprved) in the common case, while very rare cases (not presently
triggered by any REST handlers) require more queries.
One notable benefit of these changes is that functions which query
articles can now have complete control over which columns are returned.
This has not, however, been implemented yet: symbolic column groups are
still used for now.
Note that PostgreSQL still fails many tests, but the test suite runs to
completion. Note also that one line of the Database class is not
covered; later changes will eventually make it easier to cover the line
in question.
PDO does not adequately inform PostgreSQL of a parameter's type, so type
casts are required. Rather than adding these to each query manually, the
queries are instead processed to add type hints automatically.
Unfortunately the queries are processed rather naively; question-mark
characters in string constants, identifiers, regex patterns, or geometry
operators will break things spectacularly.
While the test suite passes, this commit yields a broken server: replacing ad hoc request objectss with PSR-7 ones is still required, as is emission of PSR-7 responses. Both will come in subsequent commits, with tests
Diactoros was chosen specifically because it includes facilities for emitting responses, something which is awkward to test. The end of this refactoring should see both the Response and Request classes disappear, and the general REST class fully covered (as well as any speculative additions to AbstractHanlder).
- Changed 'transform' method to use ValueInfo throughout. This fixes a number of obscure bugs
- Changed the 'add' and 'sub' methods to default to "now" rather than null. This means null passes through rather than being interpreted as the current time, to be consistent with other date tools
- Also changed the 'add' and 'sub' methods so that they operate correctly with invalid date strings
- Added tests for the class; improves #66
- Modified TTRSS tests because the "iso8601" format string in ValueInfo is different from Date's older format
Also move date formats to the ValueInfo class
Standardizing on immutables avoids any possible ambiguity in the API of the resultant value, as well as any ambiguity as to whether a DateTime output instance is the same instance or a clone (they had been clones)
This sees the addition of a dateOutFormat parameter to ValueInfo::normalize(), as well as a general simplification of how parameter binding works.
Some value type-casting results are slightly different, but this simply makes SQL statement objects consistent with the rest of the system.
The changes in this commit should make it more practical to:
- Allow the driver to decide for itself whether to try creating a PDO object if its own requirements are not met
- Have any driver use a generic schema update procedure
- Use the same constructor for native and PDO SQLite
Code coverage information is now gathered via phpdbg (a separate executable) by if available rather than xdebug, as the latter is hard to turn on and off.
A "test:quick" task has also been added to Robo, which excludes 31 tests which together account for almost two thirds of the test run time. This should pave the way for testing to be added as a commit hook for Git.
Though TTRSS itself (usually) omits items that have a counter of zero, at least one client takes this to mean the last-seen counter is unchanged, rather than zero.
Real-world use suggests 5s is inadequate. Rather than bumping up small amounts as things break, it makes more sense to wait a very long time and investigate possible long-term solutions later, once logging is implemented.
Includes PHPDoc license tag in the file-level block with accompanying copyright notice.
Also added an AUTHORS file on the off chance of outside contributions
Includes PHPDoc license tag in the file-level block with accompanying copyright notice.
Also added an AUTHORS file on the off chance of outside contributions
This introduces a data model function of unusual privilege: it can retrieve favicon URLs for any subscription, regardless of user ID. This is a single-purpose hack and its use should be avoided if at all possible.
This required adding the "notes" column to the arsse_marks table and adding same as a target value in Database::articleMark()
The Context class was also adjusted to remove the possibility of false positives in some tests
Queries for multiple specific articles are limited in size because of limits on the number of bound query parameters.
Currently this limit is somewhat arbitrarily set at 50, but it may increase.
Historically controllers would be responsible for chunking input, but this will present problems when the expected output is a result set, and of course the maintenance burden increases as the number of controllers increases.
This commit transfers the burden to the data model, and consequently introduces a ResultAggregate class which collects chunked result sets (currently only for articleList).
In the course of making these changes the mock Result class was also largely rewritten, fixing many bugs with it.
This commit does not modify the controllers nor their tests; this will be done in a subsequent commit.
- Result sets are now single-use; this is required for PDO drivers (PDO result sets are not rewindable)
- Change savepoint exceptions to be simple database exceptions; codes remain the same
This has the side-effect of removing the ability to reset a feed's title by passing null explicitly. As a non-standard behaviour it was simpler to just remove it.
This method provides generalized, consistent type casting more versatile than PHP's basic type juggling while hiding the significant complexity in achieving this.
While this commit does not change any existing code to use the new method, the intent is for both API handlers and database drivers to use the same basic rules for type conversion while still allowing for differing failure modes.
Also consolidated article star counting into a generic articleCount function which accepts a context.
This may lead to slight efficiency losses in either listing or marking (and more significant ones in counting starred), but the advantages of centralized context handling are significant with the future addition of labels and the need to count articles under various future contexts in TTRSS.
- Backend functions for adding, listing, removing, and editing (renaming) labels currently implemented
- TTRSS functions for adding (fixes #96), removing (fixes #97), and renaming (fixes #98) labels currently implemented
- Database functions now accept any input, but throw typeViolation exceptions where appropriate instead of idMissing or subjectMissing
- Added unit tests for the new Misc\ValueInfo static class
- Added ValueInfo::id() method to centrally validate database IDs, and made use of it consistently
- Made use of PHP's filter_var() function where appropriate when validating or sanitizing input
- Made the NCN protocol handler reject most invalid IDs before handing off to method handlers
- Made NCN's feedUpdate and subscriptionMove methods return 422 on invalid input
- Adjusted several tests to handler type violations
- Specifying a non-integer parent no longer silently casts to 0 or 1
- Specifying a folder ID of 0 now always converts to null automatically
- Performing both a rename and move to root in the same operation no longer results in potential duplicates
- Calling folderSetProperties with an empty data array no peforms an update; it now returns false before the update call
- Modification timestamps are now actually updated when a folder is modified
- Constraint violation exceptions triggered by code (rather than the database) now print a message
- Renaming a folder or subscription to a non-string value (e.g. an array) throws an exception rather than silently casting
- Added tests to better cover all the above
- Centralized the normalization of integers and title strings into a new ValueInfo static class
TTRSS' error message for these cases (NOT_LOGGED_IN) is not especially helpful, but that's what it returns, so that's what we should return, albeit with correct HTTP status codes.
- Added a 'user add' function to the CLI
- Removed the last trace of paths from configuration; paths must default to null and Arsse must determine sensible defaults at runtime
- Don't load a config (and possibly create a database) in CLI if a configuration is not required
- Removed the 'dbSchemaBase' config option, which is really a testing hack
- Added sample Nginx configuration
- Fixed bug in REST handler
- Readme still needs work
Previously arsse_marks had a reference to arsse_users along with arsse_articles; the former has now changed to arsse_subscriptions.
Consequently deleting a subscription now deletes marks. Tests have been simplified as a consequence.
This change was understaken to simplify counting valid marks on articles for the purposes of article cleanup; now all marks are valid marks
- Basic update service handles only one feed at a time and possibly leaks memory
- Output for REST requests is still very basic
- No avatar support
- No reporting of whether cron works
- No cleanup before or after feed updates
- use DateFormatter throughout the Feed class
- Ensure dates have TEXT affinity in SQLite, in case it matters
- Add a userPreAuth setting for when relying on the Web server to do authentication
- Changed Data::$l to Data::$lang; it's not used enough to justify the possibly confusing shortening
- Made database auto-update a general rather than per-driver setting
- Added settings for forthcoming feed fetching service
- Revamped design of Query class to be more consistent and predictable, and generally suck less
- Removed special case for Query class in Statement class
- Cleaned up database schema somewhat
- Test the values returned by articleList
- Test paged and reversed listing
- Test marking, un/read, un/starred, and both
- Nearly complete rewrite of marking queries; previous efforts simply did not work, or overzealously updated or inserted
- Testing of marking contexts still required
Fixed many errors in queries:
- Prepared statement wasn't running at all :)
- Returning feed ID rather than subscription ID
- latestEdition wasn't matching
- Fixes #55
- Included test for Context
- Adjusted Database::editionLatest() to use Context
- Adjusted NCN handler and tests accordingly
- Also refined experimental Database::articleList() method and added experimental Database::articlePropertiesSet() method
- Driver->prepare() can now take a new Database\Query object as the first parameter
- The Query class allows for building a complex query out of a main body, common table expressions, WHERE coniditions, ordering conditions, a limit, and offset
- The subscriptionList method is the prototypical example of usage
- The experimental articleList method will in time be adapted to use this as well
- A generic means of specifying a selection context is forthcoming; this would remove subscriptionList's $folder and $id parameters, for instance
- Adjusted NCN handler to use the topmost folder for folderId; suppressed the actual folder so as not to pollute output with non-standard data
- Updated tests
The previous complexity is really not required. Also modified how settingSet() works to avoid a REPLACE query, which is not compatible with PostgreSQL.
- Two items with differing IDs but identical hashes were matching in the deduplicator; they shouldn't
- They would also match in the database matcher, and shouldn't
- The second-pass database matcher was overaggressively finding items as new
Three bugs:
- The parser wrapping was generating hashes for fallback values in absence of actual values for both URL and title (which is valid if obscure RSS), URL and content, or title and content; now fallback values are detected and empty strings used instead of hashes
- The deduplicator was assuming all three hashes would always exist, which is no longer the case
- The database matcher was making the same assumption as the deduplicator
- Makes use of PHP's internal Web server to deliver expected responses from a real server
- Windows batch file can be used to run tests (Linux and Mac test runners to come later)
- Added PHPUnit to dev dependencies
- Fixes #8
- Fixes #9
- Fixes #10
- Fixes #11
- Fixes #12
- Differentiated between a missing ID which is the subject of an action (e.g. the subscription when moving a subscription) and other missing IDs (e.g. the folder when moving a subscription)
- Enforced same rules for subscription titles as for folder names, save that null is valid for subscriptions
- URLs are now matched centrally against a whitelist
- %-encoded URLs are still handled correctly
- Dispatched methods now only handle one specific task
- Filler methods (e.g. versionPOST) are no longer required
- Unhandled URLs now return 501 Not Implemented rather than 404 Not Found; this removes some ambiguity in the semantics of 404
- Implemented all but one feed-related function (it's more ofan item function)
- Fixed time conversion for input into SQL; dates in PM were previously wrong
- Added miscellaneous tentative functions to Database to help with peculiarities of NCNv1; these may change
- Tests to come soon
- functions not related to user management now have the existence of the affected user checked in the authorizer, when the affected user differs from the actor
- User::authorizationEnabled() now nests: disabling twice and then enabling once leaves the authorizer disabled
- Disabling of the authorizer is now tested
- User tests now use a partial mock instead of relying on User::authorizationEnabled()
- Added authorizer tests against a missing user
- Removed folder tests related to missing users
- Also added more subscription tests
- Introduced "strict" binding types for use when inserting into NOT NULL columns: any null value supplied is always cast to the supplied type rather than passing through
- Fixed feed updating further
- Filled out full complement of subscription manipulation functions
- made folderPropertiesSet possibly return false
