Commit fixes a bug where block references (i.e. `:block/refs`
association in the db) are lost when the page containing the referenced
block is re-loaded due to an update of its underlying file.
Description of Bug:
The bug occurs because when a file is re-loaded to the DB from disk, all existing blocks
belonging to the file are deleted via `retractEntity`, and then blocks
from the parsed file are added. If the file had only had small changes,
the new block set will be very similar to the previous one, although
with different db/ids.
However, while new blocks with "id:: " properties will assume the UUID value
of the previous version of the block, any references to that block via
UUID will *not* be restored in the DB; they are deleted with the
retractEntity command. This results in an inconsistent DB state, where references that should
exist do not.
Description of Fix:
The 'delete-blocks-fn' passed to the graph_parser has been modified as
such:
- It now accepts a list of block uuids to *retain*; graph parser will
pass the blocks parsed from the new file content.
- For any blocks which match a UUID in the retain list, instead of
deleting via retractEntity, the individual attributes are deleted via
retractAttribute (the `retract-attributes` from schema.cljs is used
for this purpose).
Why?
1. there're still a lot issues on it and there's no resource to
maintain and develop it considering we need to support it on all the
platforms.
2. it creates more bugs with Logseq Sync.
Notify users to switch to the new filename format if there're reserved
chars when syncing.
Also, this commit replace all `js/decodeURIComponent` with
`safe-decode-uri-component` to avoid UI crashes.
Fixes issues in #6970. Introduced built-in property types as there are
fewer to list as parsed than unparsed. Also removed test case that was
incorrectly introduced in #5580
Full-text search, highlights and assets alias support
It also exposes a plugin API for highlight context menu
```ts
/**
* Current it's only available for pdf viewer
* @param label - displayed name of command
* @param action - callback for the clickable item
* @param opts - clearSelection: clear highlight selection when callback invoked
*/
registerHighlightContextMenuItem: (
label: string,
action: SimpleCommandCallback,
opts?: {
clearSelection: boolean
}
) => unknown
```
* feat: new file name escaping for namespace
feat: new file name decoding back to page name
* test: file name sanitization
feat: use _0x to encode %
feat: don't create title property
test: extra URL encoding for escaped file names
fix: fit pdf prefix into new file name rules
fix: encoding rules on some characters
fix: handle the buggy file names imported by users
fix: pdf block ref failed to jump
fix: #6167
* fix: enhance backward compatibility
chore: title validation
test: fix namespace queries test
chore: use index version stored in config.edn instead of search.versions
* feat: convert old version graph mechanism
ui: file conversion UI
feat: rename files for conversion
feat: don't trigger conversion when title property is manually edited
fix: file conflict notification while renaming files on some OS
feat: re-index on update version
feat: clicking NO in the re-index dialog would update the index-ver flag to suppress the dialog
feat: use html entities for reserved char escaping
dev: remove unresolved vars & minor refactor
chore: move file name sanity from gp-util to fs-util, as it's for encoding only but not parsing
test: update file name tests to html entities rule
test: convert files from dir ver 3 for repo_tests
feat: convert Windows reserved file names
fix: save index version into idb instead of file
fix: decode uri of path while parsing files on mobile
fix: couple dir version and index version to ensure only re-index on converted dirs
feat: go back to url-encode for special chars
* chore: fix lint
chore: improve codebase to address Gabriel's comments
fix: remove file remnants on add conflict
fix: remove file remnants on rename conflict
chore: add test ns to nbb runner
Also fix typoed fn and remove unused code
* fix: issues of rebase PR6134 to master after file-sync merged
* feat: switchable filename format
* fix: use go block to replace promesa for rename all with blocking
* feat: re-index after apply rename all
* ui: file conversion enhancement
* fix: merging filename format PR with master
* fix: filename format lint & CI
* ui: filename format flow
* fix: error handling on the rare internal file path confliction case
* chore: shorten component code for files-breaking-changed
* chore: fix CI
* Minor fixes per latest code review
- Remove unused page-name-order
- Update catch usage to be consistent with what's on master
- Place state fn in right place
- Wording fixes:
- select and apply -> manual. There are no checkboxes for the user
- Update -> Edit. We use edit for all other settings button
- Alternatives to starting sentences with May. Not a common way to
start a sentence
- update outdated template comment
* ux: rename instruction update
* ux: rename instruction update (2)
* Tweak wording of conversion modal
Simplifed first paragraph and explained the page to the user in first
sentence, may isn't a common way to start sentences and updated outdated
wording
* Fix large-var warning by splitting out a piece of component
* fix: right slash on Windows; legacy format file sanitization
* fix: triple lowbar renaming fns
Co-authored-by: Gabriel Horner <gabriel@logseq.com>
* fix: keep parsing even if some whiteboards can't be transacted
* enhance: notice parse errors
* enhance: instrument parse-and-load-error
* chore: add tests for whiteboards parsing and loading
- Remove incorrect carve entry
- Move test-only code for extract-properties to test
- Remove unused if branches in extract-properties
- Move json parsing of get-references into mldoc to be consistent about
json encapsulation like other mldoc fns
- reorganize relevant unit tests
- add tests for alias, tags and comma separated config properties
- Update incorrect docstring
There're two things makes the importing slow:
1. db transact! for each file, now it's batched for 100 files.
2. block waiting for 10s before parsing so that the progress bar ui
can be changed, this is batched too for 10 files.
- Fix validator considering 'foo bar' or a number or any valid edn, a valid keyword
- Rename property keys to names as users don't know what keys are
- Also bumped nbb-logseq to latest version
- Also give property validation more product focused namespace and name
- Quoted property values were extracting refs
- Punctuation at end of tags was getting included in tag
- Property values were only getting displayed as refs/relationships
instead of full property value
- cli wasn't passing along user config
parser test caught page properties bug and dsl-query tests once written
correctly exposed another bug. Also remove unit test as they are
replaced by integration style tests in graph-parser-test
- Add tests for all *property dsl queries with and without new
config option
- Add tests for property persistence
- Add tests for property relationships
By having a specific ns for page-ref utils, our code is more readable
and intention revealing. Also found that
text/{get-page-name,page-ref-un-brackets!} was getting called in
contexts that didn't make sense e.g. query layer was checking for
markdown and org page-refs
There were a couple scattered in 2 text namespaces and there were
a couple uses of block-ref that involved escaping.
Removed block-ref-un-brackets! once it was seen as the equivalent
of an or statement
* enhance: clicking the refresh button if a query is slow
* fix: skip running slow queries if there's no need to refresh
* fix: linked reference filtering does not work on TASK items
close#1669
* fix: each block should have its own page as a reference when query page references
* fix: references
* fix: non consecutive blocks in query result and filtered linked references
* simplify filters logic
* fix: ref numbers
* Save both :block/refs and :block/path-refs for different usage
* fix: block refs
* enhance: move scheduled and deadlines to its own ns
* linked references performance tweaks
* mouse hover filters icon to expand the collapsed linked refs
* perf: react/refresh! once instead twice
* enhance: compute flashcards every hour instead of every 5s
* feat: macros as blocks
* feat: flashcards list
* fix: refed pages count
* fix: can't select in linked references
* fix: block editing on linked refs
* perf: editing in linked references
* enhance: update srs cards number when clicking flashcards
* Add a test for the case fixed in #6100
* Address feedbacks from Gabriel
* fix: Block Ref Indicator is missing from the references-blocks class
close#5375
* fix: referenced pages
* fix: page refs
* fix: Using filters pushed the title property to the second block
close#5845
Co-authored-by: Gabriel Horner <gabriel@logseq.com>
Fixes#4907. Issue was that we were losing sort order by converting a
linked/sorted map to a normal unsorted map. Also ensure
properties-order is a vec to simplify edn serialization
They were also being referred to elsewhere in block.cljs
Also moved filters to editable as that is existing behavior
Also disabled verbosity of cli tests to allow graph-parser tests to be
more readable
Peng Xiao
Gabriel Horner
Peng Xiao
Andelf
Matt Tracy
Tienson Qin
Junyi Du
Gabriel Horner
Charlie
rcmerci
8N9KT
llcc