fish shell: "shopt -s dotglob" analog

Question

we are struggling with a large migration project currently and are on the last leg of putting together the user interface.

I have a list of effectively folder paths from the old system like so:

/Programme1/Project1/WorkPackage1/Resources
/Programme1/Project1/WorkPackage1/Plans
/Programme1/Project1/WorkPackage1/Finance
/Programme1/Project1/WorkPackage1/Reporting
/Programme1/Project1/WorkPackage1/Documents
/Programme1/Project1/WorkPackage2/Resources
/Programme1/Project1/WorkPackage2/Plans
/Programme1/Project1/WorkPackage2/Finance
/Programme1/Project1/WorkPackage2/Reporting
/Programme1/Project1/WorkPackage2/Documents
/Programme1/Project2/WorkPackage1/Resources
/Programme1/Project2/WorkPackage1/Plans
/Programme1/Project2/WorkPackage1/Finance
/Programme1/Project2/WorkPackage1/Reporting
/Programme1/Project2/WorkPackage1/Documents
/Programme2/Project1/WorkPackage1/Resources
/Programme2/Project1/WorkPackage1/Plans
/Programme2/Project1/WorkPackage1/Finance
/Programme2/Project1/WorkPackage1/Reporting
/Programme2/Project1/WorkPackage1/Documents
/Programme2/Project1/WorkPackage2/Resources
/Programme2/Project1/WorkPackage2/Plans
/Programme2/Project1/WorkPackage2/Finance
/Programme2/Project1/WorkPackage2/Reporting
/Programme2/Project1/WorkPackage2/Documents
/Programme2/Project2/WorkPackage1/Resources
/Programme2/Project2/WorkPackage1/Plans
/Programme2/ Project2/WorkPackage1/Finance
/Programme2/Project2/WorkPackage1/Reporting
/Programme2/Project2/WorkPackage1/Documents

Currently these are in a csv, we need to be able to create a navigable object that a user can use to navigate through to find relevant documentation.

We have a number of issues:

1. There are 114000+ rows in the csv
2. We know the max number of subfolders and it's large (too many to code manually!).
3. There are special characters in the list, including umlauts, french accented chars + greek alphabet chars...
4. A fair number (2000+) rows of the list are longer than 400 characters..
5. We're limited also to what tools we can use. We've been playing with json/jquery/jstree/javascript/excel-vba and have had some success, but its been painful.

If anyone out there has had a similar challenge and any success I'd be interested in finding out how you went about it!

Thanks for looking.

Fohls

Solution

This is intentional. Most of time the users don't want to accidentally match the hidden files that are invisible for ls (without -a). Usually, files are hidden for a reason, not just to troll you. Also, if * would match hidden files, matching non hidden files would be too tricky.

However, unlike bash shopt -s dotglob is not needed to match hidden files. In bash, shopt -s dotglob is the only way of matching every file in the directory without accidentally matching . or ... However, fish shell can never match . or .. with globs, therefore that's not an issue (if you seriously need to match . or .. for some silly reason, just say them explicitly). Besides, fish tries to avoid having options by design, so it doesn't have dotglob.

To match every single file in the directory, you may want to use bracket expansion to detect files starting with dots, and those which aren't. {.,} is bracket expansion which matches dot that may or may not exist. The star after it matches everything. Because globs in fish cannot match . or .., the following code matches everything except for those two directories (which bash sadly matches, if you use the code below)

cat {.,}*

Comments

[SAM]

Do you need codes for a js tree?

[Robin Mackenzie]

What problem did you have with jstree?

[Fohls]

The problem we're facing is when we try to use ajax to lazy load the tree it fails with an invalid character error (due to umlauts and other special chars), if we put the json directly inside the html it will load the jstree ok using our data, but takes an age to load... When we try to use html unordered list (which has been a pain to create) it doesn't seem to lazy load either..

[Fohls]

Thanks Karim, we've finally got it working now. Turns out jstree's lazy loading actually expects snippets of the tree in the json returned. We thought the ajax call would select the children nodes we wanted by passing the parent in.... once we figured that out we created a effectively a db as you suggest and pulled back dynamic json based on the parent passed into the call...