1
zemaitis
320d

I am looking some tool to systemize large amounts of data and then be able to search across it fast. Something like my personal google+wikipedia in one. Can you guys recommend me something?

Comments
  • 0
    maybe this?
    https://cloud.google.com/bigquery
    BigQuery - Google Cloud

    Or have a look at some apache projects like Lucene or Solr
    https://projects.apache.org/project...
    Apache Projects Directory - The Apache ...

    and of course
    https://hadoop.apache.org/
    Apache Hadoop
  • 1
    none of these have user friendly GUY where everything works just out of the box.

    Im thinking personal usage, not business usage and pay 300usd a month and spent 1 week to set it up
  • 0
    Create your system
  • 0
    @pmso don't have time. I need a ready made solution, Im 100% positive that it should exist
  • 0
    @zemaitis
    I have my data classified by

    Professional and Personal

    Professional->Study and Clients

    Study->Backend,DBs,Frontend,Frameworks,Languages, etc

    In each client, I have folders with accesses, tech docs, funcional docs, etc

    In personal, I have folders for all themes I'm interested.

    When I'm searching for something, I have the notion where it is. And even when I dont know, I can use grep.

    I have almost 100GB of random docs and I can find anything I need.

    A good system isn't made in one day, it's made along the time.

    Google wasn't good in day 1. It was perfected/trained along the years.
  • 0
    @zemaitis could you please give us more info on what type of data, how big? is it local on your machine or is it over a network? on linux, windows, mac?
  • 0
    @heyheni Imagine 1000 articles like in wikipedia. And i want to be able to search across them fast.
  • 0
    @zemaitis if local use GREP Global Regular Expression (regex)

    if online have a look at the british GCHQ Intelligence services github. They open sourced their data analysis tool. It has a gui too. It's called Gaffer.

    Otherwise it's really just Solr and Elastic search as far as i know. As you need an indexer which crawles the content and makes it searchable.

    But maybe someone else knows something.
  • 0
    @heyheni grep works with searching files such as docx content?
  • 0
    @zemaitis sorry i don't know that. Google will help you.
Add Comment