Monday, April 23, 2012

"wget" gets blocked Java

Recently I faced an interesting problem. I wanted to crawl some pages from the internet and chose wget as the weapon. But, as I am a lover of Java, I wanted to call it from Java. Though I regretted a lot last hours  not following my initial plan to call it through Perl (or theoretically any scripting language) but now happy finding a hack.

Why regret? It hangs!! Just after downloading a couple of pages it sleeps to never come back alive. Well, digging out revealed, it completely occupied the stdout and the process is stuck. That means you need to flush the stdout to move forward. This can be handled by either stopping the buffering or continuously flushing the buffer!

So better would be to silent wget before it "silent"s your process. Adding the options -nv and -q, no verbose and quiet respectively, make the wget silent and your process does not get blocked.

So  at the end of the day you choose to call wget from Java, dont worry but mind to add the options -q and  -nv in your command!

Wednesday, April 11, 2012

Bag of 'Bugs'

A couple of days before, I met a coder. We started talking casually about the things and I noticed that he was carrying a small diary. Usually I am very bad at noticing unless something is really conspicuous, just in case my girlfriend is reading this. In the conversation I realised, that was the log of errors and the 'hack' he had come across so far in the life of coding. This reminded me my undergrad days when the easiest measure for geek quotient was the number of bugs one can spontaneously reciprocate. People are also used to search the error messages a lot on the web to seek the solution which I can notice from the "query suggestions" in the search engines. Okay let me say something about how the "query suggestions" work. Basically the search engines keep the track of all the queries fired and accumulate the frequencies and some advanced statistics on that. Based on these statistics, the probability of the most likely query for your already typed word/s or characters is calculated and presented in the decreasing order of probability/score as the suggestions. Mostly the query suggestions come from the previous queries and the top suggestion is the most frequent one and so on. Sometimes, these suggestions become really funny, for the fun type "nobody uses" in Google and see what are the suggestions!

Okay coming back to the original point, I think at the end of the day the knowledge is how many bugs you can solve. Here, the scope of the term "bug" is quite large. I see a strong analogy to this in the life. We call somebody 'veteran', why? Because s/he possesses a long experience of the field and intuitively can handle/hack most of the situations arise in that domain. That knowledge comes from an experience and ends up in some physical or virtual diaries somewhere which I refer as a 'bag of bugs'.