Commit Graph

298 Commits (feat/shui-button)

Author SHA1 Message Date
Tienson Qin da72d4d620 Some cleanup suggested by Gabe 2022-09-21 12:59:14 +08:00
Gabriel Horner a368cab5e4 Add ns linter to deps libs and then add+update ns docstrings 2022-09-21 10:17:15 +08:00
Gabriel Horner edd79954c1 Another test and address review feedback that shouldn't be controversial
- Added test for 71514ab36c
- Also removed dependency that is no longer used
2022-09-15 18:01:29 -04:00
Gabriel Horner ecce39d13b Misc cleanup and test improvements
- Remove incorrect carve entry
- Move test-only code for extract-properties to test
- Remove unused if branches in extract-properties
- Move json parsing of get-references into mldoc to be consistent about
  json encapsulation like other mldoc fns
- reorganize relevant unit tests
- add tests for alias, tags and comma separated config properties
- Update incorrect docstring
2022-09-14 23:26:00 -04:00
Tienson Qin b0aecd10b4 fix: keep invalid-properties for page too 2022-09-14 22:24:01 +08:00
Tienson Qin 00e13c1fc5 fix: datoms count 2022-09-14 20:45:04 +08:00
Tienson Qin 71514ab36c fix: extracting page's title property from mldoc ast 2022-09-14 20:29:44 +08:00
Peng Xiao 3d8fa09d0d fix: merge issues 2022-09-10 01:02:16 +08:00
Peng Xiao 4d86f4f53f Merge remote-tracking branch 'upstream/master' into whiteboards 2022-09-10 00:45:27 +08:00
Tienson Qin 8bd28eeeb8 fix: comma separated refs are not included in block/refs 2022-09-09 15:16:33 +08:00
Tienson Qin 79af14d75e fix: graph parser CI 2022-09-09 14:30:39 +08:00
Tienson Qin 58839059ac fix: CI 2022-09-09 14:13:35 +08:00
Tienson Qin cab5dc45a2 fix: tag parsing 2022-09-09 13:19:35 +08:00
Tienson Qin b516c1db5f Add :property/separated-by-commas option 2022-09-09 12:40:14 +08:00
Tienson Qin 2c7bae1190 fix: , and some chars shouldn't be supported for tags 2022-09-09 12:39:28 +08:00
Tienson Qin 7adf1c9d67 fix: extract ref from mldoc Tag 2022-09-09 12:39:28 +08:00
Tienson Qin 040fd5510f fix: some tests on graph_parser 2022-09-09 12:39:26 +08:00
Tienson Qin d64042df81 chore: bump mldoc for graph parser 2022-09-09 12:38:35 +08:00
Tienson Qin 2dfc3be462 fix: unused vars in graph-parser 2022-09-09 12:38:33 +08:00
Tienson Qin c376c408e0 fix: lint warnings in graph_parser 2022-09-09 12:37:57 +08:00
Tienson Qin 0266c1586b fix ci 2022-09-09 12:37:57 +08:00
Tienson Qin 9e2d6e7cfb fix: page refs 2022-09-09 12:37:57 +08:00
Tienson Qin 336fa578d5 refactor: replace directives(front matter or org) with properties 2022-09-09 12:37:55 +08:00
Tienson Qin d1d1dc50c2 wip: get references from property values 2022-09-09 12:36:54 +08:00
Gabriel Horner 51c318e2da Enable two disabled linters and bump kondo dep
All the fixes were on used-underscored-binding
2022-09-09 12:14:32 +08:00
Tienson Qin e5fd0b6529 perf: importing graph
There're two things makes the importing slow:
1. db transact! for each file, now it's batched for 100 files.
2. block waiting for 10s before parsing so that the progress bar ui
can be changed, this is batched too for 10 files.
2022-09-08 20:21:09 +08:00
Gabriel Horner 7ade955df7 Additional polish and fixes
- fix two bugs with subs - didn't have global config and missing merge
- explicit re-watch for global config
- more docs
2022-09-08 12:34:27 +08:00
Peng Xiao a58e513928 fix: whiteboard load issue on file name 2022-09-07 17:46:11 +08:00
Peng Xiao 91d63b82b9 test: failed test 2022-09-06 23:42:27 +08:00
Peng Xiao f3418a6278 fix: a potential data loss issue for whiteboard blocks 2022-09-06 23:09:26 +08:00
Peng Xiao 5e44055984 Merge remote-tracking branch 'upstream/master' into whiteboards 2022-09-02 09:46:58 +08:00
Tienson Qin 935ba7930c fix: journal titles with different capitalization in Query
results/Linked References

close #6541
2022-09-01 20:06:50 +08:00
Peng Xiao 33885e619b Merge remote-tracking branch 'upstream/master' into whiteboards 2022-09-01 10:29:29 +08:00
Charlie 01d879c18e
feat!: File Sync (#5355)
- file sync for electron/ios/android
- age encryption of both file content and file path
- massive UI enhancement
- corresponding CI tasks

Co-authored-by: llcc <lzhes43@gmail.com>
Co-authored-by: rcmerci <rcmerci@gmail.com>
Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Co-authored-by: Andelf <andelf@gmail.com>
Co-authored-by: Gabriel Horner <gabriel@logseq.com>
2022-09-01 00:31:49 +08:00
Peng Xiao f5d83483ce Merge remote-tracking branch 'upstream/master' into whiteboards 2022-08-31 10:44:18 +08:00
Peng Xiao 94e5c9d1a3 fix: whiteboard block left index on reindex 2022-08-31 10:43:11 +08:00
Peng Xiao 89c6f17283 fix: exclude whiteboard properties for auto complete 2022-08-31 10:17:19 +08:00
Gabriel Horner 9a9e793177 Update test for change in b15da3e78a 2022-08-30 11:13:38 -04:00
Tienson Qin b15da3e78a fix: page name not changed after parsing the title property 2022-08-30 12:14:11 +08:00
Peng Xiao 0ff5324c02 Merge remote-tracking branch 'upstream/master' into whiteboards 2022-08-29 10:45:15 +08:00
Gabriel Horner b47826d69d Fix page block not saving invalid-properties 2022-08-27 10:52:29 +08:00
Gabriel Horner 340fb8cf9a Invalid property fix and tests
- Fix validator considering 'foo bar' or a number or any valid edn, a valid keyword
- Rename property keys to names as users don't know what keys are
- Also bumped nbb-logseq to latest version
- Also give property validation more product focused namespace and name
2022-08-27 10:52:29 +08:00
Peng Xiao 1e607907f3 Merge remote-tracking branch 'upstream/master' into whiteboards 2022-08-26 15:46:20 +08:00
Tienson Qin 62c091840a fix: parser ci 2022-08-26 13:00:42 +08:00
Tienson Qin 133c2486c3 fix: lint warning 2022-08-26 13:00:42 +08:00
Tienson Qin abc33f213c enhance: warning on invalid properties 2022-08-26 13:00:42 +08:00
Peng Xiao 37d0ad2524 Merge remote-tracking branch 'upstream/master' into whiteboards 2022-08-24 16:50:57 +08:00
Gabriel Horner 55ccc2d17c Rename config to :rich-property-values?
Name should allow for additional functionality later
2022-08-23 09:55:15 -04:00
Peng Xiao b12fd69be0 fix: page name case 2022-08-16 18:09:13 +08:00
Peng Xiao 03ccde7008 fix: ci tests 2022-08-16 17:59:39 +08:00
Peng Xiao 384fc95231 fix: whiteboard name case issue 2022-08-16 17:52:43 +08:00
Gabriel Horner 509422901a Fix bugs with tests
- Quoted property values were extracting refs
- Punctuation at end of tags was getting included in tag
- Property values were only getting displayed as refs/relationships
  instead of full property value
- cli wasn't passing along user config
2022-08-11 17:20:47 -04:00
Gabriel Horner 8d471bd0c1 Remove uses of smart-re-find
All uses except one were locations where there were already string
checks or fns before it only worked with strings
2022-08-11 16:32:00 -04:00
Gabriel Horner 39b43b363c Fix page-properties bug and dsl-query bug
parser test caught page properties bug and dsl-query tests once written
correctly exposed another bug. Also remove unit test as they are
replaced by integration style tests in graph-parser-test
2022-08-11 16:32:00 -04:00
Gabriel Horner 0719163d30 Split out parse-property for use in dsl-query and tests
- Add tests for all *property dsl queries with and without new
  config option
- Add tests for property persistence
- Add tests for property relationships
2022-08-11 16:32:00 -04:00
Gabriel Horner 405183db09 Add config option to allow for more free text in property values
Still need to test that *property operators work correctly when this
option is on
2022-08-11 16:32:00 -04:00
Peng Xiao b9bc4130e9 fix: tests 2022-08-06 13:29:36 +08:00
Peng Xiao 37f0893b01 fix: remove some logging 2022-08-06 10:40:09 +08:00
Peng Xiao aa224fb6de fix: some displaying styles 2022-08-06 00:13:33 +08:00
Peng Xiao 1e476bd26c feat: add tabler icon extensions 2022-08-05 21:50:58 +08:00
Peng Xiao b30adea656 Merge remote-tracking branch 'upstream/master' into whiteboards 2022-08-05 13:59:06 +08:00
Peng Xiao fcd158fdc8 quick add block shape 2022-08-05 01:58:39 +08:00
Tienson Qin 59b924f25d fix: CI 2022-08-04 17:25:29 +08:00
Gabriel Horner 4ec5827902 Move block-ref fns and vars to their own ns
Similar to page-ref to keep namespaces explicit
2022-08-04 17:25:29 +08:00
Gabriel Horner e0b1f6b2de Move common uses of page-ref to its own ns
By having a specific ns for page-ref utils, our code is more readable
and intention revealing. Also found that
text/{get-page-name,page-ref-un-brackets!} was getting called in
contexts that didn't make sense e.g. query layer was checking for
markdown and org page-refs
2022-08-04 17:25:29 +08:00
Gabriel Horner 8be0deecd0 Move markdown property colons into a var 2022-08-04 17:25:29 +08:00
Gabriel Horner 001e0d302d Finish block-ref helpers
There were a couple scattered in 2 text namespaces and there were
a couple uses of block-ref that involved escaping.
Removed block-ref-un-brackets! once it was seen as the equivalent
of an or statement
2022-08-04 17:25:29 +08:00
Gabriel Horner a3d30e0b52 Move all block ref parens strings to vars
Extracted a couple fns we kept reinventing across namespaces
2022-08-04 17:25:29 +08:00
Tienson Qin a1ca6820df
Fix: broken linked references (#6105)
* enhance: clicking the refresh button if a query is slow

* fix: skip running slow queries if there's no need to refresh

* fix: linked reference filtering does not work on TASK items

close #1669

* fix: each block should have its own page as a reference when query page references

* fix: references

* fix: non consecutive blocks in query result and filtered linked references

* simplify filters logic

* fix: ref numbers

* Save both :block/refs and :block/path-refs for different usage

* fix: block refs

* enhance: move scheduled and deadlines to its own ns

* linked references performance tweaks

* mouse hover filters icon to expand the collapsed linked refs

* perf: react/refresh! once instead twice

* enhance: compute flashcards every hour instead of every 5s

* feat: macros as blocks

* feat: flashcards list

* fix: refed pages count

* fix: can't select in linked references

* fix: block editing on linked refs

* perf: editing in linked references

* enhance: update srs cards number when clicking flashcards

* Add a test for the case fixed in #6100

* Address feedbacks from Gabriel

* fix: Block Ref Indicator is missing from the references-blocks class

close #5375

* fix: referenced pages

* fix: page refs

* fix: Using filters pushed the title property to the second block

close #5845

Co-authored-by: Gabriel Horner <gabriel@logseq.com>
2022-08-04 13:03:12 +08:00
Peng Xiao ab6eec5fc3 Merge remote-tracking branch 'upstream/master' into whiteboards 2022-08-03 14:12:10 +08:00
Peng Xiao 035b96e70a feat: add whiteboard toggle 2022-08-02 10:55:19 +08:00
Gabriel Horner 49fbdd0244
Merge pull request #6214 from logseq/fix/hidden-properties-in-autocompletion
Fix: Property completion for hidden properties and property ordering
2022-08-01 14:42:06 -04:00
Peng Xiao 45272ef52b Merge remote-tracking branch 'upstream/master' into whiteboards 2022-08-01 10:22:15 +08:00
Gabriel Horner 42b7f5d189 Add two editable org built-in properties
Also ported old filetags property tests from b991599244.
2022-07-30 06:33:37 +08:00
Gabriel Horner b1c98ec09c Fix property ordering for large property maps
Fixes #4907. Issue was that we were losing sort order by converting a
linked/sorted map to a normal unsorted map.  Also ensure
properties-order is a vec to simplify edn serialization
2022-07-28 15:46:18 -04:00
Peng Xiao 66a08f8916 Merge remote-tracking branch 'upstream/master' into whiteboards 2022-07-26 09:31:48 +08:00
charlie bb2362b5d6 improve(ux): remove :file-path from hidden page properties 2022-07-25 23:38:57 +08:00
Peng Xiao 751df3b97d fix: filter out none shapes for tldraw 2022-07-25 20:51:43 +08:00
Peng Xiao b83f4fb1a2 Update extract.cljc 2022-07-24 12:25:37 +08:00
Peng Xiao 754b952173 fix 2022-07-24 12:15:58 +08:00
Peng Xiao cefe46072e fix: whiteboard path-refs 2022-07-24 10:26:20 +08:00
Peng Xiao 6bf4b05141 feat: pasting shapes into outliner 2022-07-23 23:31:42 +08:00
Peng Xiao d919c6c363 fix: block name should always be lowercase 2022-07-23 22:44:11 +08:00
Peng Xiao 4f2626e297 fix: block back ref uuid 2022-07-23 22:32:07 +08:00
Peng Xiao 45798ba714 feat: displaying whiteboard text shape in outliner 2022-07-22 14:47:57 +08:00
Peng Xiao 4d42877f3d fix: parsing/saving issues on block/refs 2022-07-22 10:29:24 +08:00
Gabriel Horner 48be2c9990 Update to latest nbb-logseq 2022-07-21 14:32:40 -04:00
Gabriel Horner c8864e777d Fix page-properties not using :property-pages/* config 2022-07-21 13:59:16 -04:00
Peng Xiao 8eefd44540 fix: shape uuid should be persisted 2022-07-21 22:57:06 +08:00
Peng Xiao 26f04b073b Merge remote-tracking branch 'upstream/master' into whiteboards 2022-07-21 10:24:48 +08:00
Peng Xiao 1ce9017c5d feat: first working poc for whiteboard as page 2022-07-20 02:04:18 +08:00
8N9KT f79888b4d8 raname ":exclude-in-graph" to ":exclude-from-graph-view"
- raname ":exclude-in-graph" to ":exclude-from-graph-view"
- add ":exclude-from-graph-view" in "editable-built-in-properties"
2022-07-19 16:55:04 +02:00
Peng Xiao f4715f59fd rename whiteboard pages 2022-07-19 00:22:19 +08:00
Peng Xiao 2bb7ef5919 Merge remote-tracking branch 'upstream/master' into whiteboards 2022-07-18 22:50:43 +08:00
Gabriel Horner 7370716215 Move editable linkable built-in props to var
They were also being referred to elsewhere in block.cljs
Also moved filters to editable as that is existing behavior
Also disabled verbosity of cli tests to allow graph-parser tests to be
more readable
2022-07-15 10:41:35 -04:00
llcc 54a7e61b26 Revert: don't parse some special property key values as pages
Produced two many extra pages.
2022-07-15 16:11:11 +08:00
llcc cd6c355e8d fix testings 2022-07-15 15:52:00 +08:00
llcc 1b5627ed4d add :icon :aliases to editable-built-in-properties 2022-07-15 15:52:00 +08:00
llcc 4ea1cbadaf don't parse some special property key values as pages
e.g. numbers, boolean
2022-07-15 15:26:09 +08:00
llcc f371d99034 don't parse value of built-in property keys as pages 2022-07-15 15:26:09 +08:00
Andelf 5f74ba6907 fix: allow edit icon:: property 2022-07-15 15:26:09 +08:00
Gabriel Horner a0087d7792 Add config options to disable property pages and to exclude certain ones 2022-07-15 15:26:09 +08:00
Gabriel Horner de61b28000 Add additionable built-in properties that shouldn't have pages created
Introduced hidden naming as that is a more accurate name for its usage.
There may be a possible bug with existing built-in-properties but not
familiar with all cases here and don't want to introduce more bugs
before release
2022-07-15 15:26:09 +08:00
Gabriel Horner abd7e0becb Add 3 missing built-in-properties
Also fix carve linter and make nbb test invocation easier
2022-07-15 15:26:09 +08:00
Gabriel Horner 50bce350de Don't auto-create pages for built-in properties
Fixes #5963
2022-07-15 15:26:09 +08:00
Peng Xiao 2d4e0bef18 fix ends-with nil issue 2022-07-13 19:28:22 +08:00
Peng Xiao 0b1119a61f fix tests 2022-07-13 14:46:38 +08:00
Peng Xiao dbb49fe8d1 lint 2022-07-13 14:33:45 +08:00
Peng Xiao deb0268ac5 fix graph-parser tests 2022-07-13 14:28:11 +08:00
Peng Xiao a84ec7a5f3 fix block uuid 2022-07-13 14:20:30 +08:00
Peng Xiao d33d1cfede wip add more tests 2022-07-13 03:20:22 +08:00
Peng Xiao 06bfdee7b4 support parsing edn file as pages 2022-07-10 01:39:34 +08:00
Peng Xiao d3dae5fddc Merge remote-tracking branch 'upstream/master' into whiteboards 2022-07-07 14:07:35 +08:00
Tienson Qin ea5fac1a70 fix: replace some invalid chars for property keys 2022-07-07 01:42:16 +08:00
Tienson Qin 264801fedf fix: verify property key && make property values string when searching 2022-07-07 01:42:16 +08:00
Tienson Qin e8023b9246 fix: graph parser ci 2022-07-07 01:42:16 +08:00
Tienson Qin 49acb54178 feat: add properties as linked references 2022-07-07 01:42:16 +08:00
Tienson Qin 08263d6f96 fix: wrong file path name when renaming 2022-07-06 00:03:41 +08:00
Tienson Qin 73a202b3c2 fix: redirect to home when deleting a page 2022-07-04 20:42:41 +08:00
Tienson Qin 1b2ba25521 fix: rename namespace page results in lower-case page 2022-07-04 20:12:46 +08:00
Peng Xiao fb832d2493 fix: indexing whiteboard page files 2022-07-04 00:41:29 +08:00
Peng Xiao 4de59c6d63 wip whiteboard page 2022-07-04 00:00:08 +08:00
Peng Xiao 35d00eda74 code cleanup 2022-07-03 15:55:01 +08:00
Peng Xiao 26ad627661 Merge remote-tracking branch 'upstream/master' into whiteboards 2022-07-03 12:46:36 +08:00
Junyi Du 50c9e69c21 test: update page number due to CANCELED / CANCELLED built-in pages added 2022-06-30 16:26:56 +08:00
Junyi Du cd661e0eb0 enhance: built-in canceled page 2022-06-30 16:26:56 +08:00
llcc bba9c935e0 feat: open plaintext assets files in logseq 2022-06-29 10:34:52 -04:00
Junyi Du 0cdacc35e2 feat: import from logseq edn
feat: import edn with provided uuid

feat: overwrite page uuid; use properties in content

feat: error handling for importing

feat: support json import

chore: fix lint by splitting setup ui
2022-06-24 17:11:24 +08:00
Peng Xiao 36513b7391 Merge remote-tracking branch 'upstream/master' into whiteboards 2022-06-20 15:12:07 +08:00
Gabriel Horner 69ba15252e cli/parse-graph returns ast data as well
This data was not returned and is valuable for CI jobs
and possibly other functionality
2022-06-20 10:06:08 +08:00
Peng Xiao 6b5b1940d3 Merge remote-tracking branch 'upstream/master' into whiteboards 2022-06-14 10:17:21 +08:00
Gabriel Horner d0755ef161 Update dependencies in deps
Also fix conditional workflows that didn't trigger if changes were made
to them
2022-06-10 15:39:11 +08:00
Gabriel Horner ee9f390761 Localize bb-tasks for deps
Deps need to be independent to not introduce testing bugs.
These bb tasks are good candidates to move into bb-tasks gitlib
2022-06-10 15:39:11 +08:00
Gabriel Horner 17d2462343 Move rules into db dep as it's useful for external apps
- rules is already used by logseq-query and will be useful for
  nbb-logseq
- Updated db jobs to run in deps/db by default. Less typing but
  more importantly discourages independent libraries from depending on
  external scripts.
- Moved lint-rules task into db since it is db specific now
- Also remove nbb-logseq from top-level as all nbb compatibility is in
  deps.
2022-06-10 15:39:11 +08:00
Gabriel Horner 6608537909 Move logseq.graph-parser.db* namespaces to their own dep
Also add its own CI file
2022-06-10 15:39:11 +08:00
Gabriel Horner 01b28e26ee Exceptions during block extraction shouldn't delete data
This will prevent future occurrences of #5548
2022-06-08 13:19:50 +08:00
Gabriel Horner 5188c86547 Add tests for bug fix in #5553
Test normal and buggy case. Put these tests in graph-parser as
it allows us to test more user facing assertions and is easier to setup
2022-06-08 13:19:50 +08:00
Peng Xiao 5c2e644a43 Merge remote-tracking branch 'upstream/master' into whiteboards 2022-06-07 10:52:11 +08:00
Andelf 333017235b fix(graph-parser): avoid parsing illegal id 2022-06-05 15:12:01 +08:00
Tienson Qin 78d1b17743 fix: ignore parsing for :file-path 2022-06-05 12:39:12 +08:00
Gabriel Horner 29f73549aa Moved nbb tasks to another repo
They are useful outside of logseq development and don't need to be
embedded in the Logseq app
2022-06-01 01:17:29 -04:00
Gabriel Horner 83a4e62bfd Merge branch 'master' into enhance/graph-parser-part-four 2022-05-30 10:18:34 -04:00
Gabriel Horner 70c0708f09 Add nbb:watch task and example
Also update to latest clj-kondo
2022-05-27 20:42:17 -04:00
Gabriel Horner 7e5ba8d705 Upgrade to latest clj-kondo dep and pods
- Remove cljs 1.11 workarounds since latest includes those symbols
- Unique duplicate ns which intermittently show up in script-vars
  despite using :lang :cljs
- Able to use :format :progress since failing bug no longer shows up
- Remove unnecessary ignore and unused garden config
2022-05-27 10:07:22 -04:00
Gabriel Horner b142327491 Finish up logseq.graph-parser
- Parser now parses all graph files like the app does, not just pages and journals.
  This required extracting another fn from repo-handler
- Add and tweak CI steps that are specific to graph-parser. All
  namespaces in this library are checked for nbb compatibility
- Cleaned up parser cli API so only one fn is needed for scripts
- Tests were updated to match new parsing behavior
- large_vars.clj can run with a smaller max-line-count after only refactoring two fns
- Add docs
2022-05-27 00:54:30 -04:00
Gabriel Horner 1e299052ce Setup cljs tests for graph-parser
- Also reuse docs graph test assertions to ensure main app
and graph-parser are in sync.
- Removed promesa from extract tests
since they are no longer async
- Added cljs and npm deps that were missing and caught by cljs tests
2022-05-26 18:04:45 -04:00
Gabriel Horner 15b8e5e9df Setup CI jobs that are only for graph-parser
Also move all non graph-parser fns to frontend thanks to carve lint
2022-05-26 18:04:45 -04:00
Gabriel Horner 4813d903d5 Move graph-parser to its own dir 2022-05-26 18:04:45 -04:00