DailyDogo 875 🐶

#buildinpublic #zknstuff The library is closing soon and I’m spent mentally for the moment. Time to take the rest of the Sunday off!

#buildinpublic #zknstuff I did it. Around 200 notes have been reviewed, filed away, linked to and from, some of them tagged, renamed, deleted (very few). Most of them were how-to notes, some were ideas, some were project journals, some just held some information like an address, a measurement, etc.

#buildinpublic #zknstuff Still 66 files to go. An observation: I need to do focussed reviews, that is, reviews that only tackle part of the system, more often. That I can review files in this way is actually very helpful to make those notes I took time to write more useful and discoverable.

#buildinpublic #zknstuff Still 136 Files to review in my Inbox and I might not do anything else after this, actually, since this is going to take a moment still. But I’m very happy that the simple note review plugin works now as expected.

#buildinpublic #zknstuff I think that either simple note review or the underlying dataview plugin got a fix, because it now actually works as expected?! I’m reviewing, moving and lightly editing the notes in my Inbox folder and try to think a little how to review my readwise highlights next.

#buildinpublic #zknstuff Even though I had abandoned the simple note review plugin since it didn’t work very well and the spaced repetition plugin seemed more stable I have now reverted to the former. Main reason: Having different review queues, since having just one giant queue didn’t cut it.

#buildinpublic #zknstuff I’m trying to figure out how to use these library Sundays the best at the moment and my idea was to use the spaced repetition plugin for it. I have done that a handful of weeks now but with so many notes it is kind of unsatisfying. So let’s think about the review process a little, cleaning up what is there and maybe finding a new way to do things.

Today I’m spending time in the library working on my notes. I thought I’d try to do a #buildinpublic kind of thing, since it was fun the last couple of days to post about what I’m doing as regards to #mbsync. so I’m going to try this here, too. Hashtags will be #buildinpublic and #zknstuff.

#buildinpublic #mbsync That’s it for today. Lots of little things done. Learned some stuff and realized a lot of things about how to structure the sync state. This’ll be the most important part of the app, so it’s alright if it takes some time and some attempts to get right.

#buildinpublic #mbsync Alright. The test for a post that is unchanged does the thing and I already created a similar test for a newly created post. However I don’t like how the sync status is saved in the db. Or rather how different kinds of logic are reduced to only the sync state. We need to differentiate between the state of the sync itself (pending/synchronized/error) and what we sync (creation/update/deletion). At least I think it’d be nice to be explicit about it. Makes it more testable. So we need two fields in the db on every post:

• sync_status: pending, synchronized, error, none
• sync_intention: create, update, delete, none

#buildinpublic #mbsync So. Me the last few hours:

I’m not distracted, I’m on a side quest. — https://beige.party/@RickiTarr/112264314660393539

Where was I? Oh yes. Refactoring/Writing a test for the markForSync method. I have the starting point for the test, which creates a post using a factory and writes that same data to disk. Now we need to run the post through our method and see that the posts sync_status is not marked for creation/update/deletion.

DailyDogo 874 🐶

#buildinpublic #mbsync Ugh. And even more problems. I use the laravel validated dto package, because I like working with DTOs and getting validation “for free” is nice. However:

If a post doesn’t have a category the mb api returns an empty array. Fine. I can validate this as 'categories' => ['present', 'array'],, but interestingly the DTOs saves this internally as null. And guess what? categories in not nullable anymore when I try to save a post with empty categories to the db. Hmpf.

#buildinpublic #mbsync Discoverd another problem: I was allowing the categories field (json in the db) in my Post model to be null. Turns out if you do that you can’t cast that json to array through eloquent. So I changed that in my migrations, but this means that the assumption you can write a post to the db with categories being null doesn’t hold anymore. So more yak shaving…

#buildinpublic #mbsync I’m much more familiar with codeception and my brain is kinda mushy so figuring out how to do things in PHPUnit took its time. However, I refactored that storePost method now and have a test (which I wrote first or at least started to write first) to make sure it does what I want it to do. Now back to testMarkForSync.

#buildinpublic #mbsync After a late combined breakfast/lunch back at it. Still writing the testcase for storePost. I had to think about how to structure tests that rely on test data or rather needs a directory to write to. I generally like to mirror the structure of the app in my tests, so that I can find the things I need easily. I have now introduced a testData directory where I can write to/from the filesystem. It looks like this:

#buildinpublic #mbsync Well, that worked. We just had to add:

public function posts(): \Illuminate\Database\Eloquent\Relations\HasMany
    {
        return $this->hasMany(Post::class);
    }

to the Blog model.

Now we can do the following in our test:

$blog = Blog::factory()->has(Post::factory()->count(1))->create();
$post = $blog->posts->first();

That post now has some fake data as defined in my PostFactory.

Time to write this to disk. We could use PostFilePersistorService::storePost. This method makes a couple of assumptions about where to write data and I would like it to just write where I want it to. So I will refactor the path generation logic out of storePost. In the future it should be handed in via a parameter. This is a big change but we’re the only developer and in the “break lots of things” phase, so I feel this is fine.

There is only one problem: We again have no test! So what I’ll do is make the other test pass, add a TODO to it and create a test for storePost. First things first.

#buildinpublic #mbsync Well, it seems that my models are incomplete. I am missing a relationship between a blog and its posts. This is needed so that my factory can create some posts to test with.

#buildinpublic #mbsync No tests, alright. What do we want to test here? If we call markForSync() a changed post, a deleted post, a new post and an unchanged post should all be detected correctly. So let’s start with the simple case that a post in the db and in the filesystem are the same.

#buildinpublic #mbsync Continuing from yesterday. I will rewrite the markForSync method so it iterates over the posts in the db instead of the files to be able to detect deleted files as well. Just have to check if there are any tests to work with, or if I have to add them first.

#buildinpublic #mbsync Well. Looks like I will not continue today…

DailyDogo 873 🐶

#buildinpublic #mbsync Will continue but we have stuff to do in the city.

#buildinpublic #mbsync It seems that I have a column in the posts table called sync_status that is enum('pending','synced','deleted') so it seems to me I need to set this to deleted if the file is missing, which implies that instead of iterating over the blog post files and marking the posts in the db I need to iterate over the blogposts in the db since otherwise I miss the deleted files. Does this make sense?

• Files:
  • 1
  • 2
  • 3
• Posts in DB
  • 1
  • 2
  • 3

I was checking the files against the db to find new and updated posts, but I’m now going to check the db against the files instead. I should still be able to find new/updated posts, but also posts that are missing from the filesystem.