1
1
Fork 0
mirror of https://code.mensbeam.com/MensBeam/Arsse.git synced 2024-12-23 17:12:41 +00:00
Commit graph

370 commits

Author SHA1 Message Date
J. King
dd24246f49 Finished last of feedUpdate tests; implemented NCN API functions
- Fixes #26
- Fixes #27
2017-06-03 17:34:37 -04:00
J. King
0514321a76 Save enclosures and categories properly in feedUpdate 2017-06-03 14:08:33 -04:00
J. King
0ab6210214 Fix feed ID discovery so it doesn't short-circuit the item loop 2017-06-03 13:43:58 -04:00
J. King
ba61ca2b2c Add category list to feed items 2017-06-03 11:16:26 -04:00
J. King
c6b7e653c5 Tests for feedUpdate and feedListStale
The update process neglects to handle enclosures; this will have to be fixed
2017-06-02 19:16:36 -04:00
J. King
81419452b5 Added topmost folder to output of subscriptionList. Fixes #54
- Adjusted NCN handler to use the topmost folder for folderId; suppressed the actual folder so as not to pollute output with non-standard data
- Updated tests
2017-06-01 18:12:08 -04:00
J. King
ba6b5dee85 Simplify in-database settings
The previous complexity is really not required. Also modified how settingSet() works to avoid a REPLACE query, which is not compatible with PostgreSQL.
2017-06-01 16:24:11 -04:00
J. King
719290c82a Remove dependency on Webmozart's Glob outside of testing
Native glob is effectively tested via use of unmodified Lang throughout non-Lang tests
2017-06-01 13:50:46 -04:00
J. King
5b8f23441c Feed tests: article matching
These should be the last required tests for the Feed class
2017-05-30 20:18:04 -04:00
J. King
3e5343e3e8 Feed tests: last set of next-fetch tests, with fixes 2017-05-27 23:14:43 -04:00
J. King
7656de716b Feed tests: fetching and parsing 2017-05-27 18:15:52 -04:00
J. King
c800852f3a Remove stubs to support non-HTTP authentication
Also removes authPreferHTTP setting
2017-05-26 13:06:06 -04:00
J. King
6771e8916a Fixed more Feed bugs
- Two items with differing IDs but identical hashes were matching in the deduplicator; they shouldn't
- They would also match in the database matcher, and shouldn't
- The second-pass database matcher was overaggressively finding items as new
2017-05-23 22:15:57 -04:00
J. King
fd7d1c3192 Fixes to Feed class for bugs uncovered by initial deduplication tests
Three bugs:
- The parser wrapping was generating hashes for fallback values in absence of actual values for both URL and title (which is valid if obscure RSS), URL and content, or title and content; now fallback values are detected and empty strings used instead of hashes
- The deduplicator was assuming all three hashes would always exist, which is no longer the case
- The database matcher was making the same assumption as the deduplicator
2017-05-23 20:39:29 -04:00
J. King
ed3e3c12d6 More Feed tests: more cache header handling 2017-05-22 13:01:38 -04:00
J. King
adfb22e922 macOS fixes 2017-05-21 21:49:54 -04:00
J. King
97a8f473a7 More Feed tests; Linux fixes
- Adjusted some namespace case
- Included Linux test runner; should also work with macOS
- Made the PHP server run in the background on Windows
2017-05-21 19:51:03 -04:00
J. King
590abaf0ef Start on tests for Feed
- Makes use of PHP's internal Web server to deliver expected responses from a real server
- Windows batch file can be used to run tests (Linux and Mac test runners to come later)
- Added PHPUnit to dev dependencies
2017-05-21 17:16:32 -04:00
J. King
88c0f4986c Implement NCN API v1-2 feed handling: tests
- Fixes #8
- Fixes #9
- Fixes #10
- Fixes #11
- Fixes #12

- Differentiated between a missing ID which is the subject of an action (e.g. the subscription when moving a subscription) and other missing IDs (e.g. the folder when moving a subscription)
- Enforced same rules for subscription titles as for folder names, save that null is valid for subscriptions
2017-05-21 10:10:36 -04:00
J. King
054200dfc0 Fix potential of erroneous matching of URL containing encoded #. 2017-05-20 08:57:24 -04:00
J. King
0972cff660 Completely revamped NCNv1 REST handler
- URLs are now matched centrally against a whitelist
- %-encoded URLs are still handled correctly
- Dispatched methods now only handle one specific task
- Filler methods (e.g. versionPOST) are no longer required
- Unhandled URLs now return 501 Not Implemented rather than 404 Not Found; this removes some ambiguity in the semantics of 404
2017-05-19 23:52:26 -04:00
J. King
4a816f827b NCNv1 feed calls and other changes
- Implemented all but one feed-related function (it's more ofan item function)
- Fixed time conversion for input into SQL; dates in PM were previously wrong
- Added miscellaneous tentative functions to Database to help with peculiarities of NCNv1; these may change
- Tests to come soon
2017-05-18 23:03:33 -04:00
J. King
0bc2841837 Last of subscription tests
- Also tweaked SQL dateformat() function to output proper HTTP dates
- Also introduced method to set a default output date format
2017-05-18 13:21:17 -04:00
J. King
ef3cb4fa54 Test unread counts in subscriptionList 2017-05-16 22:19:40 -04:00
J. King
f635155687 Basic tests for subscription list
- Need entries and read marks to fully test
- Added assertion type to test result subsets
- Fixed custom subscription titles
2017-05-14 23:03:48 -04:00
J. King
acbb254bfb Tests for removing subscriptions 2017-05-11 23:20:10 -04:00
J. King
98c950de58 Removed most calls to userExists()
- functions not related to user management now have the existence of the affected user checked in the authorizer, when the affected user differs from the actor
- User::authorizationEnabled() now nests: disabling twice and then enabling once leaves the authorizer disabled
- Disabling of the authorizer is now tested
- User tests now use a partial mock instead of relying on User::authorizationEnabled()
- Added authorizer tests against a missing user
- Removed folder tests related to missing users
- Also added more subscription tests
2017-05-11 18:00:35 -04:00
J. King
8ebcb01cb5 Made out-of-order transaction handling more predictable 2017-05-07 18:27:16 -04:00
J. King
2083c6e397 Change transactions to auto-rollback on exceptions 2017-05-06 12:02:27 -04:00
J. King
c2a7ad7b19 Optimize initial feed update 2017-05-04 22:43:18 -04:00
J. King
db132c0c24 Various changes to Database
- Introduced "strict" binding types for use when inserting into NOT NULL columns: any null value supplied is always cast to the supplied type rather than passing through
- Fixed feed updating further
- Filled out full complement of subscription manipulation functions
- made folderPropertiesSet possibly return false
2017-05-04 19:12:33 -04:00
J. King
9c7f4710aa Fix feed fetching
There are almost certainly other bugs; proper tests forthcoming
2017-05-04 14:42:40 -04:00
J. King
9dccdb32a7 Fold feedAdd into subscriptionAdd
A feed will enver been added independently of a subscription
2017-05-04 11:13:24 -04:00
J. King
717e74fb2e Refactored special case for testing 2017-05-03 20:00:29 -04:00
J. King
8376cc9cb9 Mark articles as unread when they are updated 2017-05-01 18:29:07 -04:00
J. King
7546e3b11b Add the ability to throw exceptions in a feed update if desired
This is useful when adding a new feed
2017-04-30 18:36:31 -04:00
J. King
43f73b5d71 Make better use of prepared statements when updating feeds 2017-04-30 18:14:11 -04:00
J. King
3bc9082cfa Integrated adaptive update intervals Fixes #51 2017-04-30 17:54:29 -04:00
J. King
3b4d79abc8 Implement second part of adaptive update interval; improves #51
Implements part of algorithm used when feed has been updated
2017-04-27 09:47:40 -04:00
J. King
2e64e60f2e Implement part of adaptive update interval; improves #51
Implements part of algorithm used when a feed has not been updated; this is much simpler than when a feed has been modified
2017-04-24 21:51:56 -04:00
J. King
f842439b01 Deduplicate feed items within the feed itself 2017-04-23 13:12:33 -04:00
J. King
93c010d3d5 Move feed item change detection to Feed class 2017-04-22 23:40:57 -04:00
J. King
b1b96bb806 Completed refactoring of feed updates
- In-feed deduplication still to do
- Content scraping still to do
- Lots and lots of testing still required
2017-04-20 21:59:12 -04:00
J. King
119d42907e More feed update refactoring
Still very much incomplete: in its present form it would yield many false duplicates
2017-04-15 22:07:22 -04:00
J. King
4e57e56ca7 Start on refactoring of feed processing 2017-04-13 22:17:53 -04:00
J. King
557d17ef5d Add rollbacks in error cases 2017-04-09 18:15:00 -04:00
J. King
a67fe30408 Fix whitespace
Also fixed my editor so tabs won't happen again!
2017-04-06 21:41:21 -04:00
J. King
b02abec250 Added test for recursive binding
Also fixed a bug uncovered by the test
2017-04-06 21:24:26 -04:00
J. King
818707b067 Fix adding a subscription (again)
Also ensured HTTP last modified date is parsed properly.
2017-04-06 20:50:47 -04:00
J. King
e6f2f22888 use list() instead of extract()
Improves clarity where Database::generateSet() is actually used at the expense of clarity within the method itself
2017-04-06 13:29:39 -04:00
J. King
929d763981 Simplify the creation of arbitrary UPDATEs
The type parameters of Db\Driver::prepare() and the parameters of Db\Statement::run() can now be arrays, which will be iterated over recursively to bind scalar values to the SQL statement.

This simplifies the construction of arbitrary UPDATE statements (the WHERE clause no longer needs to be taken into account) and should make it clearer what is happening in these cases.

It should also simplify the creation of IN() clauses down the road if they become necessary.
2017-04-06 11:02:47 -04:00
Dustin Wilson
25aeaf4801 Grab favicon from site's URL, not the feed's
• Changed Feed to where the favicon is gotten when parsing so the
feed’s site URL is used instead of the feed’s for favicons. Some feeds
don’t have the same domain.
• Fixed a bug in Database.
2017-04-02 21:23:15 -05:00
J. King
74ae1106a4 Implement NCN API v1-2 server version reporting
- Fixes #29
- Yes, it's trivial, but now it's done!
2017-04-02 21:49:37 -04:00
J. King
9cbfa378bc Implement NCN API v1-2 folder deleting/renaming
- Fixes #5
- Fixes #6
- Rewrote the NCNv1 dispatcher to better handle URL edge cases
2017-04-02 21:34:30 -04:00
J. King
19abce85c3 Implement NCN API v1-2 folder creation
- Fixes #4
- Tests for failure modes still required
2017-04-02 12:14:15 -04:00
J. King
842e277d43 Implemented NCN API v1-2 folder list
- Fixes #2
- Also re-organized REST handling
2017-04-01 23:06:52 -04:00
J. King
dc9f5e545e Fix adding a feed
Also start on handling the v1-2 API
2017-04-01 15:42:10 -04:00
J. King
1e1b848c62 Remove root field from folders table
The field is no longer required with the use of recursive common table expressions, and  presents a possible loss of referential integrity
2017-04-01 14:49:31 -04:00
J. King
a111bcc231 Folder get/set prop funcs and other changes
- Simplified folder removal; now properly relies on foreign keys for dependency resolution
- simplified *propertiesSet() methods by offloading input validation and query building to a generic function
- Implemented function to get the properties of a single folder (useful for internal use)
- Implemented a function to set the properties of a folder
2017-04-01 10:27:26 -04:00
J. King
2344911f78 Tests for removing folders 2017-03-31 18:48:24 -04:00
J. King
458eaac5c6 Tests for listing folders
- Fixed checking for existence of parent before listing
- Changed Result->getAll() to return an empty array rather than null for an empty result set
2017-03-31 17:42:28 -04:00
J. King
b68cea1188 Tests for adding folders 2017-03-31 15:27:59 -04:00
Dustin Wilson
bc863ae935 Updated Database->updateFeeds()
• Added category updating in Database->updateFeeds()
• Made Database->updateFeeds() fail quietly when exceptions are
occurred, logging them in the database instead of failing.
• Changed the categories table name from arsse_tags to arsse_categories.
2017-03-30 09:42:37 -05:00
J. King
dbd735f1f3 Full set of tests for user-related Database funcs
- Fixed a few inconsistencies and bugs along the way
- User manager tests need some equivalent fleshing out
2017-03-29 23:41:05 -04:00
Dustin Wilson
9338219a68 Improved UA String
• Added version constant in bootstrap
• Made the user agent string output information in a format similar to
a web browser with software version along with OS information and
architecture
2017-03-29 09:27:54 -05:00
Dustin Wilson
85800b111e New UA string
• Now Arsse exposes itself properly
• Conf->userAgentString exists to allow for specifying the string
through configuration
2017-03-28 18:19:12 -05:00
J. King
f902346b6c Eliminated passing of RuntimeData instances
- RuntimeData has now been replaced by a single static Data class
- The Data class has a load() method which fills the same role as the constructor of RuntimeData
- The static Lang class is now an instantiable class and is a member of Data
- All tests have been adjusted and pass
- The Exception tests no longer require convoluted workarounds: a simple mock  for Data::$l suffices; Lang tests also use a mock to prevent loops now instead of using a workaround
2017-03-28 18:50:00 -04:00
Dustin Wilson
7236020b8b Changed "NewsSync" to "Arsse" 2017-03-27 23:12:12 -05:00
J. King
b8f9b6070e Experimental method to remove a folder 2017-03-27 08:39:24 -04:00
Dustin Wilson
ce0584e7f8 Added Feed Updating
• Started implementing feed updating (Database->updateFeeds())
• Moved hashing to the Feed object, now done when parsing
• Moved adding of articles to the database to its own method
(Database->articleAdd())
2017-03-26 15:16:15 -05:00
J. King
8c76c22d74 Implement retrieving folder list in Database 2017-03-24 22:39:18 -04:00
J. King
254ffe7b48 Don't enforce 100-character limit on folder names 2017-03-24 13:26:13 -04:00
J. King
8b50297e6d Complete tests for NCN version list
- Fixes #47
- Implemented an AbstractHandler class with a generic URL parser
2017-03-24 13:16:34 -04:00
J. King
25d9158171 Fix output of NCN version list
List is supposed to be contained in a JSON object, not a bare list
2017-03-19 21:54:28 -04:00
J. King
536fa5c4fd Implement NextCloud News version detection
- Improves #47
- Still needs wrapping to actually output to clients
2017-03-19 21:50:00 -04:00
J. King
9a8def03c2 Fill in more of the REST handler.
Not yet functional.
2017-03-18 22:30:36 -04:00
Dustin Wilson
3f61921b97 Added picoFeed wrapper
• Implemented a simple wrapper for picoFeed which fixes the id problems
and keeps error handling within its own class
• Updated Database.php to use the new class
• Replaced mentions of ownCloud with NextCloud in the schema
• Added hashes to schema for identification and change detection;
removed NextCloud hash and fingerprint; removed enclosure and category
hashes
2017-03-18 11:01:23 -05:00
J. King
b821d728e4 Add rest dependency; fix SQLite function def 2017-03-13 23:22:20 -04:00
J. King
6413d3a489 Re-order SQLite3 driver initialization
Should handle error reporting better
2017-03-13 19:13:06 -04:00
Dustin Wilson
bb8af5a62c Added Custom Functions into Db/SQLite3/Driver
• Created Db/SQLite3/CustomFunctions to house the dateFormat custom
function and potentially others in the future.
• Fixed a typo in Db/SQLite3/Driver
2017-03-13 13:33:31 -05:00
J. King
8c268581fb Properly handle adding a folder
Also added test file for now
2017-03-09 22:41:11 -05:00
J. King
18c8076a0f Database update tests 2017-03-09 21:39:42 -05:00
J. King
689c1c77d0 Test for advisory locking 2017-03-09 17:14:26 -05:00
J. King
f0663e99c3 Add tests for transactions and savepoints
Existing code had subtle bugs; using increment and decrement operators actually makes things easier to understand (for once)
2017-03-09 16:36:33 -05:00
J. King
485400df2b Rename Result->get() to Result->getRow
It should be slightly clearer what it does
2017-03-09 15:01:18 -05:00
J. King
bdf3182305 Add result->lastId method; cleanup
- Result->lastId() added as MySQL and PostgreSQL have equivalent functionality
- Adjusted tests accordingly
- Cleaned up Database class to make use of this and getAll()
2017-03-09 14:48:42 -05:00
J. King
a3e2da1d36 Flesh out most database exceptions
- Fixes #46
- Some exception messages are tentative pending testing
2017-03-08 12:29:22 -05:00
J. King
7c1df71acd Make db throw specific exceptions
Needs testing and fleshing out; not all exception codes and messages have been defined
2017-03-08 09:55:16 -05:00
J. King
755c976aa4 Remove extraneous files 2017-03-08 08:21:48 -05:00
J. King
fa247b34f6 Reduce the number of database exception types 2017-03-08 08:19:42 -05:00
J. King
7687109132 Reorganize Db namespace; alter User ns to match 2017-03-07 18:01:13 -05:00
J. King
37dad63dee Impelemented Result->getAll()
- Fixes #45
- Renamed getSingle to getValue to avoid possible confusion
- Added test to ensure getValue() always returns the first datum of each row rather than going column-to-column
2017-03-06 16:34:38 -05:00
J. King
1dab15c20e Fix SQLite3 driver name 2017-03-04 20:03:41 -05:00
J. King
eeb1818bb5 Still more database changes
- Restructured tests
- Localized driver name for SQLite driver (fixes #37)
- Ensured that binding type definitions are required
2017-03-03 13:20:26 -05:00
J. King
2b7a236147 More database fixes
Now able to add feeds to the database
2017-03-02 22:43:59 -05:00
J. King
007e3747ef Various database fixes
Authentication appears to be broken with a real database
2017-03-02 20:47:00 -05:00
J. King
0c410fcf50 More binding tests and related changes
- Introduced abstract Statement class to hold common methods
- Common methods currently consist of a date formatter and type caster
- Moved binding tests to a trait for reuse with future drivers
2017-03-02 18:42:19 -05:00
J. King
1529fc367a Partial proper tests for parameter bindings
Null and (>0) integer tested so far

Many related changes to accomodate the necessary type juggling (required for databases other than SQLite).
2017-03-02 14:19:16 -05:00
J. King
337b2cf90c Remove last vestiges of PDO accomodation 2017-03-02 09:04:04 -05:00
J. King
f19f683e38 Make it possible to redefine Statement type binds 2017-03-01 23:12:42 -05:00
J. King
b4d178c33f Remove most PDO stuff
PDO will not be supported: the feature-set is less than any of the common native APIs, so why bother?
2017-03-01 22:47:51 -05:00
J. King
176247894d Tests for Db\ResultSQLite3 2017-03-01 19:00:14 -05:00
J. King
766f2b65a4 Cover unknown/invalid user levels 2017-02-28 11:19:33 -05:00
J. King
7ae3c978b3 Full set of tests for authorization
Invalid/unknown rights levels still need better testing.
2017-02-28 10:52:02 -05:00
J. King
e4852b581a Start on tests for authorization 2017-02-27 23:04:13 -05:00
J. King
798d7e7ab7 Added synthetic tests for external User drivers
Fixed two typos in external driver handling
2017-02-26 22:06:14 -05:00
J. King
1834cb9963 Last of the basic tests for the User class 2017-02-25 12:59:39 -05:00
J. King
be9ebf9ca1 Remove authz from User driver; moved to main class 2017-02-22 23:22:45 -05:00
J. King
8a0d021622 More User tests and resultant fixes 2017-02-20 19:04:08 -05:00
J. King
7785eb072b Complete rewrite of User class and other changes
- User-related database methods will now throw User\Exception upon errors
- Internal userAdd method can now generate random passwords
- Pursuant to above, dependency on password genrator has been added, and password-related methods now return strings instead of booleans
- User class methods now all explicitly follow different branches for internal/external/missing implementations
- various User class methods now perform auto-provisioning of the internal database when external implementations report success on users not in the database
- Tests have been adjusted to account for the above changes
- Lots is probably still broken
2017-02-20 17:04:13 -05:00
Dustin Wilson
26989facfe More PicoFeed Integration Bug fixes
• Fixed a bug where the feed object’s variable was being reused as the
id for adding feeds to subscriptions
• Modified inline documentation
• Added a TODO for populating the articles table when adding a new feed
2017-02-20 11:58:26 -06:00
Dustin Wilson
cb664eab03 Minor fixes to PicoFeed integration 2017-02-19 18:33:03 -06:00
Dustin Wilson
e5d825d360 Started integration of PicoFeed
• Integrated PicoFeed into Database->subscriptionAdd
• Added exception handling for feeds
• Added static method for formatting SQL dates into Db/Common
2017-02-19 16:02:03 -06:00
J. King
e6feb8de8d First basic test for User class
As the User class depends on the database and this has yet to be tested (though I'm fairly certain it works), the mock driver also acts as a mock of the required database functions, with both instances sharing a common storage structure. Later test series should separate the two.
2017-02-19 00:22:16 -05:00
J. King
574388665a Tidy up tests and User stuff
- Skeleton of mock internal driver
- Skeleton of test suite
- Re-arranged lots of code
- Made drive name localized (improves #37)
2017-02-17 19:22:50 -05:00
J. King
8db31cf3e4 Tweaked Lang; added comments and tests
Tweaks:
- get() method can now report loaded and well as wanted locale
- msg() passed without vars still gets formatted to catch malformed strings
- set('en', false) followed by set('en', true) will now immediately load
- Lang::$synched was not getting set to true properly

Tests:
- Added test for get()
- Added test for malformed strings (exception code was missing)
- Added test for missing strings
- Added test for strings taking variables not being passed any variables
2017-02-16 17:50:34 -05:00
Dustin Wilson
c5fac33398 Converted all hard tabs to soft tabs 2017-02-16 14:29:42 -06:00
J. King
8afa1cb5bb Class names from classes themselves in defaults 2017-02-16 15:14:15 -05:00
J. King
c8274eca74 Moved base exception code to abstract class
This allows differentiating specific and general exceptions in tests; a library-wide trap would have to trap AbstractException
2017-02-11 14:56:02 -05:00
J. King
c213b5928d move code out of vendor tree into lib 2017-02-11 14:48:48 -05:00