site  contact  subhomenews

SFSget now much faster

March 31, 2020 — BarryK

Those who have used SFSget, see "sfsget" icon on the desktop, will have experienced a big delay while the online SFS repository is probed. This was done with 'wget' "spider" mode, searching for all files under the "sfs" folder. This is very slow.

Now, the "spider" search is eliminated, as I am uploading a file, named 'sfs-repo-file-list', that lists all the files under the "sfs" folder. All that SFSget has to do is download that file. Here it is:

Two files have been edited, /usr/local/sfsget/sfsget and /usr/local/sfsget/filterpkgs. The latter has a fix for properly handling different SFS repositories.

If anyone wants to setup their own SFS repository, it needs to be added to the 'SFS_REPOS' variable in /root/.packages/DISTRO_PET_REPOS. This is the current assignment:


You would need to have an upload script that creates file 'sfs-repo-file-list'. This is my script:

#200331 upload a list of files.

[ ! $1 ] && exit 1
[ ! -d "$PATH1" ] && exit 1

echo -n 'Enter the password for '
[ "$SSHPASS" == "" ] && exit
echo " ...ok, ${SSHPASS}"
export SSHPASS

#create a date file for sfs uploads
DATE="$(date +%Y%m%d%H%M)" #ex: 201908121739
case "$PATH1" in
#200331 have to keep this file for older versions of easyos...
echo "$DATE" > ${PATH1%/sfs/*}/sfs/date-last-upload
#seems have to upload this separately...
echo "Uploading 'date-last-upload' file..."
rsync -cpt -e "sshpass -e ssh -p 22" ${PATH1%/sfs/*}/sfs/date-last-upload${PATH1%/sfs/*}/sfs
#200331 upload file-list...
find ${PATH1%/sfs/*}/sfs -mindepth 2 -type f > ${PATH1%/sfs/*}/sfs/sfs-repo-file-list
PTN="s%^${PATH1%/sfs/*}/sfs/%%" #ex: easyos/oe/pyro/kernel_5.2.21-p1_src.sfs
sed -i -e "$PTN" ${PATH1%/sfs/*}/sfs/sfs-repo-file-list
echo "Uploading 'sfs-repo-file-list' file..."
rsync -cpt -e "sshpass -e ssh -p 22" ${PATH1%/sfs/*}/sfs/sfs-repo-file-list${PATH1%/sfs/*}/sfs

rsync --exclude-from=rsync-exclusions --exclude=synchronize --delete -rltzvc -e "sshpass -e ssh -p 22" ${PATH1}/${PATH1}

SFS files need to be in an appropriate path. for example:

Locally, you would have something like "www/amd64/packages/sfs/easyos/debian/buster" with the SFS files (and .specs and .png files, see the 'dir2sfs' script). The above script would be at "www/remote-sync", and you would sync the buster SFS files like this:

# ./remote-sync amd64/packages/sfs/easyos/debian/buster

Some more reading: 

Tags: easy