Re: Speeding up IO
I can't really read your programs intension or structure but here are a
few pointers:
- Don't use Vector, they have been replaced a long time ago by the
faster ArrayList.
- For gods sake, don't use meaningless variables like that.
- For performance enhancements, try to use the java.nio/memory mapped file.
- Never catch Exception and then do nothing, at the very least throw a
stacktrace.
- Try to use another mechanism than all those if's, a lookup table or
switch would yield better results.
- Use a profiler to find out where the bulk of the time is spent.
- For gods sake, you make it look like Java is procedural and not OO.
/Casper
ruds wrote:
Hi,
I have to work on file which are in GB's. They contain some id's and
their related data. These Id's repeat after few hundred or more lines
but with different data. I want to find these Id's and operate on the
data i.e for each id i want to compare data with similar id down the
file.
For this i'm writting following code...
class Risk
{
static String file;
String str,id,sid="0",eid,s,s1,s2,str1,str2,tok1,tok2;
int
a,i=0,j=0,k,l,sc,line,no,flag=0,flag1=1,flag2=1,start=0,end=0,x,y,cnt;
double
val1=0.0,val2=0.0,val3=0.0,val4=0.0,val5=0.0,val6=0.0,val7=0.0,val8=0.0,val9=0.0,val10=0.0;
FileReader fr,fr1,fr2;
BufferedReader br,br1,br2;
FileWriter fw,fw1,fw2;
StringTokenizer st;
int[] subcase=new int{1,2,3,4};
Vector qid=new Vector();
qid.add(100);
qid.add(101);
qid.add(102);
qid.add(103);
qid.add(104);
double qmax[]=new double[11];
String[][] quad=new String[50][30];
public void compare(String infile) throws IOException
{
cnt=0;
for(i=0;i<qid.size();i++)
{
no=0;
fr=new FileReader(infile);
br=new BufferedReader(fr);
while((str=br.readLine())!=null)
{
no++;
if((str.startsWith("$"))||(str.startsWith("-CONT-")))
continue;
else
{
s2=str.substring(0,10);
if(s2.equals(qid.elementAt(i)))
{
cnt++;
start=no;
end=no+29;
quadarray(infile,start,end);
}
if((cnt==sc) && (i><qid.size()))
{
System.out.println("qid="+qid.elementAt(i));
cnt=0;
writesubcase1();
}
}
}
fr.close();
}
public void quadarray(String ifile,int start,int end) throws
IOException
{
try
{
fr1=new FileReader(ifile);
br1=new BufferedReader(fr1);
line=0;
k=0;
x=0;
while((str1=br1.readLine())!=null)
{
line++;
if((line>=start) && (line<end))
{
if(j==0)
quad[j][k]=str1;
if((k==3) || (k==17) || (k= ))
{
val1=Double.parseDouble(str1.substring(18,36));
if(val1>qmax[x])
{
qmax[x]=val1;
x++;
}
}
if((k==5) || (k==8) || (k==22) || (k==25))
{
val2=Double.parseDouble(str1.substring(54,72));
if(val2>qmax[x])
{
qmax[x]=val2;
x++;
}
}
if((k==11) || (k==14) || (k==28))
{
val3=Double.parseDouble(str1.substring(36,54));
if(val3>qmax[x])
{
qmax[x]=val3;
x++;
}
}
k++;
}
}
}
catch (Exception e)
{ }
}
public void writesubcase1() throws IOException
{
x=0;
try
{
fw=new FileWriter("Result.txt",true);
for(y=0;y<30;y++)
{
if((y==0) || (y==1) || (y==2) || (y==4) || (y==6) || (y==7)
|| (y==9) || (y==10) || (y==12) || (y==13) || (y==15) || (y==16) ||
(y==18) || (y==19) || (y==21) || (y==23) || (y==24) || (y==26) ||
(y==27))
{
fw.write(quad[0][y]+"\n");
continue;
}
else
{
if((y==3) || (y==17) || (y= ))
{
s=quad[0][y];
fw.write(s.substring(0,28)+qmax[x]
+s.substring(37)+"\n");
x++;
continue;
}
if((y==5) || (y==8) || (y==22) || (y==25))
{
s=quad[0][y];
fw.write(s.substring(0,64)+qmax[x]+"\n");
x++;
continue;
}
if((y==11) || (y==14))
{
s=quad[0][y];
fw.write(s.substring(0,46)+qmax[x]
+s.substring(55)+"\n");
x++;
continue;
}
if(y==28)
{
s=quad[0][y];
fw.write(s.substring(0,46)+qmax[x]+"\n");
x++;
break;
}
}
}
fw.close();
}
catch(Exception e)
{}
}
public static void main(String []args)
{
Risk r=new Risk();
file=args[0];
r.compare();
}
Can you tell me how can i speed up IO coz its really taking hell lot
of time to process data.
Thankx in advance.>