Thursday, May 31, 2012

External process calls from java: Runtime.exec() hangs

My recent blog was about calling wget from Java which was talking about a temporal solution. The solution was just to mute the output of the external process so that the output buffer never gets exhausted. Unfortunately, it doesn't solve the bigger problem of calling external processes from Java through Runtime.exec() which can very easily exhaust the system buffer and cause a deadlock at Process.waitFor().

I read a hell lot of articles and spent a couple of hours, looking for the solutions. I got many but all of them were not solving my problem. Then I checked one of the solutions and discussed it at #fedora-java at freenode to see if the problem is really with Java in Fedora! With the help of mizdebsk I was able to spot the concurrency issue in the code which I have posted here. The conclusion after the incorporation of the following code was the external process took some abnormally more time at some point in the program but in the end it was successful to finish rather than waiting forever. I have just defined the block to handle output here, similar block for ErrorStream should be included and if your process expects some input then it should be added too.


class ExecCommand {
private Semaphore outputSem;
private String output;
private Semaphore errorSem;
private String error;
private Process p;

private class OutputReader extends Thread {
 public OutputReader() {
  try {
  outputSem = new Semaphore(1);
  outputSem.acquireUninterruptibly();
  } 
  finally {
   outputSem.release();
  }
 }

 public void run() {
  try {
  StringBuffer readBuffer = new StringBuffer();
  BufferedReader isr = new BufferedReader(new InputStreamReader(p
   .getInputStream()));
  String buff = new String();
  while ((buff = isr.readLine()) != null) {
   readBuffer.append(buff);
   System.out.println(buff);
  }
  output = readBuffer.toString();
  outputSem.release();
  } catch (IOException e) {
  e.printStackTrace();
  }
 }
}

public ExecCommand(String command) {
 try {
  p = Runtime.getRuntime().exec(makeArray(command));
  new OutputReader().start();
  new ErrorReader().start();
  p.waitFor();
 } catch (IOException e) {
  e.printStackTrace();
 } catch (InterruptedException e) {
  e.printStackTrace();
 }
}

public String getOutput() {
 try {
   outputSem.acquireUninterruptibly();
 } 
 finally {
   outputSem.release();
 }
 String value = output;
 outputSem.release();
 return value;
}

private String[] makeArray(String command) {
 ArrayList<String> commandArray = new ArrayList<String>();
 String buff = "";
 boolean lookForEnd = false;
 for (int i = 0; i < command.length(); i++) {
  if (lookForEnd) {
  if (command.charAt(i) == '\"') {
   if (buff.length() > 0)
   commandArray.add(buff);
   buff = "";
   lookForEnd = false;
  } else {
   buff += command.charAt(i);
  }
  } else {
  if (command.charAt(i) == '\"') {
   lookForEnd = true;
  } else if (command.charAt(i) == ' ') {
   if (buff.length() > 0)
   commandArray.add(buff);
   buff = "";
  } else {
   buff += command.charAt(i);
  }
  }
 }
 if (buff.length() > 0)
  commandArray.add(buff);

 String[] array = new String[commandArray.size()];
 for (int i = 0; i < commandArray.size(); i++) {
  array[i] = commandArray.get(i);
 }

 return array;
}
}


Add this class to your code and use the following code to call it for you.
String command = "your external command command";
         
ExecCommand e;
e = new ExecCommand(command);

This takes care of almost all the things, parsing the command line and printing the output to the console. Moreover, sometimes I have noticed that adding a sleeptime in the ExecCommand constructor also helps.
p = Runtime.getRuntime().exec(makeArray(command));
  new OutputReader().start();
  new ErrorReader().start();
  int sleeptime = 1000; // Try to make an educated guess 
                        // about the sleeptime
  Thread.sleep(sleeptime);
  p.destroy();


This block of code is the corrected version the code given <Here>. Basically the changes are related to outputSem.acquire() to outputSem.acquireUninterruptibly() and adding the finally block. Without them it was easily leading to a deadlock.

Monday, April 23, 2012

"wget" gets blocked Java

Recently I faced an interesting problem. I wanted to crawl some pages from the internet and chose wget as the weapon. But, as I am a lover of Java, I wanted to call it from Java. Though I regretted a lot last hours  not following my initial plan to call it through Perl (or theoretically any scripting language) but now happy finding a hack.

Why regret? It hangs!! Just after downloading a couple of pages it sleeps to never come back alive. Well, digging out revealed, it completely occupied the stdout and the process is stuck. That means you need to flush the stdout to move forward. This can be handled by either stopping the buffering or continuously flushing the buffer!

So better would be to silent wget before it "silent"s your process. Adding the options -nv and -q, no verbose and quiet respectively, make the wget silent and your process does not get blocked.

So  at the end of the day you choose to call wget from Java, dont worry but mind to add the options -q and  -nv in your command!

Wednesday, April 11, 2012

Bag of 'Bugs'

A couple of days before, I met a coder. We started talking casually about the things and I noticed that he was carrying a small diary. Usually I am very bad at noticing unless something is really conspicuous, just in case my girlfriend is reading this. In the conversation I realised, that was the log of errors and the 'hack' he had come across so far in the life of coding. This reminded me my undergrad days when the easiest measure for geek quotient was the number of bugs one can spontaneously reciprocate. People are also used to search the error messages a lot on the web to seek the solution which I can notice from the "query suggestions" in the search engines. Okay let me say something about how the "query suggestions" work. Basically the search engines keep the track of all the queries fired and accumulate the frequencies and some advanced statistics on that. Based on these statistics, the probability of the most likely query for your already typed word/s or characters is calculated and presented in the decreasing order of probability/score as the suggestions. Mostly the query suggestions come from the previous queries and the top suggestion is the most frequent one and so on. Sometimes, these suggestions become really funny, for the fun type "nobody uses" in Google and see what are the suggestions!

Okay coming back to the original point, I think at the end of the day the knowledge is how many bugs you can solve. Here, the scope of the term "bug" is quite large. I see a strong analogy to this in the life. We call somebody 'veteran', why? Because s/he possesses a long experience of the field and intuitively can handle/hack most of the situations arise in that domain. That knowledge comes from an experience and ends up in some physical or virtual diaries somewhere which I refer as a 'bag of bugs'.

Sunday, February 5, 2012

How you want to be treated?

The first point that arrives in picture when you take any service which can be from simple dining out to a long international trip : "How would I be treated?". More than five years ago, I remember when I underwent a summer training of customer dealing, I was taught "Remember! You are the face of the company. You portray the image of the company to the customer. So think of your actions and words." This is so true. Each good and bad experience with the companies around reminds me the same lines. I try to see the same point from the both perspective an employee and a customer.

The rule of the world is very interesting: the customer is an employee somewhere and deals with fellow customers directly or indirectly. This makes the situation very easy to understand. For example, we all know that how the bureaucracy sucks sometimes, but only one person sitting on the other side of the table has the power to turn our experience upside down or otherwise. If that person "treats the people as he wants to be treated" the world of bureaucracy will be much simpler. Unfortunately, this is somewhat related to corruption but I do not want to talk about it at the moment. On the contrary, the inverse is also true that "I am treated very badly and the same will now be reflected in my services!" :) So these are directly related nodes of the chain "the customer and the employee".

I hope to see the chain in proper direction and would end with related lines: "Treat the people around the same way you want to be treated!"

Saturday, November 19, 2011

Do you have 'Frames'?

Yesternight I was watching Vinod Dua Live on NDTV, well its my one of the favourites and I feel bad when I miss it. He has the vision and is outspoken though he does his research before making any point. Basically an ideal piece of impressive and effective journalism. He speaks only for 15 minutes and it feels like he has conveyed something that some ordinary person will take whole day to convey.

This is just a recent show video for your reference (Click on the image):



Personally I don't want to be judgemental about his reports nor am here to increase his fan following because it is none of my business though I am a big fan of the show.

Okay coming back to the title: 'Frames' a concept I was taught by somebody in the school that "If you don't want/like something then just put a denial frame in your mind and then you will realize, that thing is no longer a problem for you. Basically your mind learns to ignore that thing very easily."

We have lots and lots of frames in our mind. I don't want to enlist them because they also include some of the social stigmas. Now, why I introduced Mr. Dua's show? Whenever I watch his show and listen to his sharp comments, I feel he tries to break the frames in the minds of India - the frames which are very bad but we are not ready to break them. He tries to make the point and if a citizen tries to incorporate some of them to which she herself agrees, significant improvement would be visible in the society. Basically I don't like to give any advice but I am extremely sorry to say this piece of advice here as I couldn't help it. I think those frames are so strong in our minds that even such strong opinion and sharp journalism is weak to break them.

"Isi Umeed me ki hamara aanewala kal aaj se behtar ho" I sign out!