Linux za Sve forum

drone je napisao/la:

sumski je napisao/la: Al mislim da niste skužili poantu - naravno, file kojem (djelomice) znam ime nebudem tražil s nepomukom, al file koju u sebi sadrži određene riječi - da:

Find i grep/locate mi nebudu izlistali žanrove/albume/izvođače il bilo koji drugi metadata podatak:

Mislim da smo ipak skužili poantu, ali ti možda nisi skužio na što mislimo. Find i locate neću koristiti za te stvari kao niti bilo što drugo jer to imam u glavi i u odgovarajućim direktorijima prikladnih naziva (npr. .../metal/melodeath/Insomnium/...)

Ok, možda netko ne želi ići previše u dubinu poddirektorija nego mu je jednostavnije svu muziku trpati u isti, pa onda to indeksirati s nekim programom. Ja ne volim takav pristup. Kad znam gdje mi je što, ne trebaju mi takvi žderači resursa. Za sve ostalo tu je Google

Da ne bi bilo zabune, nemam ništa protiv da netko koristi nešto što mu olakšava život, samo kažem da postoje različiti pristupi.

So Nepomuk is for you if you:
- Think that organising your files only with subdirectories and filenames is not informative and efficient enough.
- Have large collections of pictures and/or music and you want to perform searches among them.
- Have large collections of pictures and/or music and you want to get information of the files without opening it (i.e. who is on a picture, which one was good etc.).
- Keep different versions of the same file or folder on your hard drive and you want to have information available which specifies them.
- Work with more files distributed in different folders over your hard drive and you don’t want to have so many file manager tabs opened.
- Want to perform indexed searches for file contents over all files on your hard drive.
- Download lots of files from the internet where you need to remember the download-URL (i.e. for citations in scientific papers)
- Want to store more information to files you downloaded.
- Have large documents or papers where you want to rate them and remember the important parts.


Nepomuk is not for you if you:
- Feel completely satisfied organising your hard drive by files and folders
- Work frequently on different computers and operating systems (because the KDE Nepomuk database is not transferrable to Microsoft Windows)

sumski je napisao/la:

> On July 25, 2012, 7:06 p.m., Sebastian Trueg wrote:
> > OK then, ship it. Strigi needs some serious fixing or replacement I fear.

I have plans to replace it. Give me some time.

Hey everyone

This month I'm focusing on the file indexing part of Nepomuk, and right now
it takes forever for Strigi to index all my files. Additionally, it doesn't
do a very good job of it. I have tons of mp3 files whose metadata is not
correctly outputted by Strigi. This obviously makes Nepomuk not index those
files.

I realize this is a big change, but I would like to stop using Strigi. Here
is why -

* Doesn't always handle PDFs, Microsoft Document Formats
* Doesn't always handle ID3 tags properly
* Seeks into video files thereby slowing down the extraction
* Implements its own parsers for archives and utf handling
* Goes berserk handling some large video files
* Large code base
* Difficult to contribute to
* Very little documentation
* Un-maintained
* We have hacks on the Nepomuk side to get the correct types
* We use KDE's mimetype detection instead of Strigi


I'm not the only one with this problem. We already have another project
called the nepomuk-metadata-extractor [1] which implements the following
indexers -
* PDF ( Poppler Based )
* Audio Files ( Uses Taglib )
* Videos ( Only based on the file name )

I would like to move these indexers into nepomuk-core, and create light
wrappers to handle whatever file types are missing. Just to be clear, I am
not proposing a fancy plugin based architecture like Strigi. We would just
be detecting the mimetype using KMimeType. It would then call the
appropriate indexing class (if one exists) which would populate the
SimpleResourceGraph or it would just add the appropriate rdf types.

I've created a simple page listing some of the common file formats [2] and
how we would handle them. I obviously still need to figure out how we would
handle document files. I would love to reuse the code in Calligra + Okular
instead of rolling our own. Apart from that it seems fairly straight
forward.

What do you guys think?

I don't think this entire port should take me more than a week.

Another update

I've pushed my changes into the feature/newIndexer branch. If someone could
review it, it would be nice.

The current architecture consists of 2 queues - BasicIndexingQueue and
FileIndexingQueues.

The BasicIndexing queues just extracts the mimetype, stat results and url.
On my system, with the latest Soprano, I manage around 10 files per second.
This queue is NOT throttled in any way, and make virtuoso peak around 70%
of one cpu. I'm still working on reducing this. I would ideally like this
part to not be noticeable. Even if it is working on full speed.

The FileIndexingQueue calls the 'nepomukindexer' process which extracts the
actual metadata from the file. It only works when the system is IDLE. This
is monitored using the KIdleTime, which is not that great, since I could
have left a compiling job and during that time I don't want the file
indexing to start. Ditto when watching an HD movie.
...
5. Plugins - They are only 5 plugins so far, and I have no plans of writing
any more. They are extremely simple to write, and my time is better spent
doing other things. I think this is an amazing place to get people
interested. So, we need to finalize (4) so that I can blog about it and
start talking about it.
...
That's about it