Cluster resources and usage by the Orthologs project

The Orthologs project consumes a lot of computing power, here are the statistics and total times contributed by each machine for all the successful runs doing the all-against-all alignments. The actual resources consumed are higher due to failures, genomes which had to be redone, which are not counted here, and other computations which are less significant in time. These times were collected on Mon Apr 6 17:13:26 2009

job statistics by machine
machine	jobs	hours	Mcell/s	Tcell	#proc	Ghz

deimos	31229	24610	26.98	2390.79 (0.17%)	2	1.00
enigma	24099	23894	24.20	2081.52 (0.15%)	2	1.60
linneus1	36916	42910	18.78	2901.23 (0.21%)	2	0.80
linneus2	38896	44822	18.94	3056.94 (0.22%)	2	0.80
linneus3	40820	45544	18.34	3006.94 (0.22%)	2	0.80
linneus4	38693	44768	18.92	3049.57 (0.22%)	2	0.80
linneus5	39025	45201	18.83	3064.56 (0.22%)	2	0.80
linneus6	37839	43957	18.71	2960.82 (0.22%)	2	0.80
linneus7	37629	43749	18.73	2949.77 (0.22%)	2	0.80
linneus8	37333	43856	18.42	2908.90 (0.21%)	2	0.80
linneus9	39086	45203	18.85	3067.28 (0.22%)	2	0.80
linneus10	37777	44117	18.72	2972.53 (0.22%)	2	0.80
linneus11	23807	31661	15.54	1770.81 (0.13%)	2	0.80
linneus12	38610	44234	18.92	3012.23 (0.22%)	2	0.80
linneus13	35539	40340	18.69	2713.73 (0.20%)	2	0.80
linneus14	37677	43401	18.94	2959.82 (0.22%)	2	0.80
linneus15	36704	43412	18.45	2882.75 (0.21%)	2	0.80
linneus16	37824	44158	18.65	2965.01 (0.22%)	2	0.80
linneus17	32412	43620	16.34	2566.00 (0.19%)	2	0.70
linneus18	33354	44467	16.43	2630.42 (0.19%)	2	0.70
linneus19	32399	43324	16.24	2533.57 (0.19%)	2	0.70
linneus20	31523	42333	16.16	2463.45 (0.18%)	2	0.70
linneus21	14076	19345	13.44	936.11 (0.07%)	1	0.80
linneus22	35086	27304	26.41	2595.94 (0.19%)	2	1.80
linneus23	64563	40070	34.75	5012.66 (0.37%)	2	2.00
linneus24	58004	39363	31.67	4487.95 (0.33%)	2	1.80
linneus25	62726	42544	31.45	4817.15 (0.35%)	2	1.80
linneus26	58067	39819	31.39	4500.04 (0.33%)	2	1.80
linneus27	59433	40769	31.37	4604.77 (0.34%)	2	1.80
linneus28	63462	40404	33.89	4929.82 (0.36%)	2	1.80
linneus29	52481	36805	30.76	4075.90 (0.30%)	2	1.80
linneus30	60357	41288	31.33	4656.27 (0.34%)	2	1.80
linneus31	59320	40830	31.34	4605.87 (0.34%)	2	1.80
linneus32	59713	40956	31.28	4612.56 (0.34%)	2	1.80
linneus33	57947	39901	31.28	4493.56 (0.33%)	2	1.80
linneus34	59014	40648	31.28	4577.49 (0.33%)	2	1.80
linneus35	57761	40016	31.16	4489.35 (0.33%)	2	1.80
linneus36	57553	39455	31.02	4406.03 (0.32%)	2	1.80
linneus37	59226	40903	31.18	4591.92 (0.34%)	2	1.80
linneus38	56455	39313	30.90	4373.38 (0.32%)	2	1.80
linneus39	59515	41123	31.21	4621.04 (0.34%)	2	1.80
linneus40	46812	27249	36.25	3556.33 (0.26%)	2	1.80
linneus41	57389	39561	31.16	4437.71 (0.32%)	2	1.80
linneus42	53136	36840	31.00	4111.96 (0.30%)	2	1.80
linneus43	63852	39008	36.17	5079.37 (0.37%)	4	2.20
linneus44	44930	29198	31.26	3285.44 (0.24%)	2	2.00
linneus45	27144	17057	34.08	2092.60 (0.15%)	1	2.07
linneus46	30452	18674	34.54	2321.91 (0.17%)	1	2.07
linneus47	20893	12894	32.50	1508.32 (0.11%)	1	2.07
linneus48	709216	268718	91.49	88505.15 (6.47%)	14	3.20
linneus49	654751	253113	97.29	88654.25 (6.48%)	14	3.20
linneus50	18251	7660	54.83	1511.91 (0.11%)	1	3.20
linneus51	987605	354670	198.80	253826.21 (18.55%)	28	2.80
linneus52	19203	8260	69.06	2053.57 (0.15%)	4	2.40
linneus53	482358	179754	70.65	45718.47 (3.34%)	28	2.80
linneus54	38566	21381	42.49	3270.41 (0.24%)	4	1.80
linneus55	30366	15075	108.25	5874.87 (0.43%)	4	1.80
linneus56	1030329	292780	295.00	310937.85 (22.73%)	32	3.40
linneus57	650999	179146	430.67	277751.38 (20.30%)	32	3.40
linneus58	22713	8444	229.25	6969.23 (0.51%)	4	1.80
linneus71	650	266	468.87	448.93 (0.03%)	8	2.00
lumberjack	11005	7817	34.62	974.30 (0.07%)	2	2.50
mars	9609	10830	18.25	711.59 (0.05%)	2	0.65
phobos	31223	24374	27.28	2393.67 (0.17%)	2	1.00
ru0	21402	26892	15.34	1484.71 (0.11%)	2	0.87
ru6	55938	34729	32.66	4083.61 (0.30%)	2	2.00
saturn	189307	102687	39.02	14424.07 (1.05%)	8	2.08
sim0	79938	102122	18.54	6816.72 (0.50%)	8	0.70
sim64	26047	18154	33.10	2163.14 (0.16%)	2	1.80
xibalba165	28641	26807	24.80	2393.68 (0.17%)	2	1.00
xibalba166	26014	24658	24.27	2154.31 (0.16%)	2	1.00
xibalba167	28460	26568	24.53	2345.83 (0.17%)	2	1.00
xibalba168	32446	29752	25.06	2683.59 (0.20%)	2	1.00
xibalba169	33040	29964	25.16	2713.64 (0.20%)	2	1.00
xibalba170	33362	30299	25.23	2752.01 (0.20%)	2	1.00
xibalba171	31951	28699	25.44	2627.94 (0.19%)	2	1.00
xibalba172	32471	29180	25.46	2674.34 (0.20%)	2	1.00
xibalba173	33086	29968	25.35	2734.81 (0.20%)	2	1.00
xibalba174	32679	29573	25.40	2704.18 (0.20%)	2	1.00
xibalba175	29481	26903	25.06	2427.34 (0.18%)	2	1.00
xibalba176	21000	20886	22.98	1727.73 (0.13%)	2	1.00
xibalba177	32144	28950	25.42	2649.77 (0.19%)	2	1.00
xibalba178	32177	29251	25.18	2651.43 (0.19%)	2	1.00
xibalba179	32639	29365	25.39	2684.45 (0.20%)	2	1.00
xibalba180	29132	26373	25.67	2436.97 (0.18%)	2	1.00
xibalba181	30728	26036	26.47	2481.24 (0.18%)	2	1.00
xibalba182	33062	29813	25.39	2724.87 (0.20%)	2	1.00
xibalba183	33104	29875	25.44	2735.77 (0.20%)	2	1.00
xibalba184	32168	29307	25.16	2654.28 (0.19%)	2	1.00
xibalba185	32396	29460	25.20	2672.53 (0.20%)	2	1.00
xibalba186	33256	30206	25.19	2739.08 (0.20%)	2	1.00
xibalba187	30809	27353	25.89	2549.19 (0.19%)	2	1.00
xibalba188	19333	17131	27.10	1671.05 (0.12%)	2	1.00
xibalba189	32677	29406	25.54	2703.44 (0.20%)	2	1.00
xibalba190	33383	30192	25.36	2756.53 (0.20%)	2	1.00
xibalba191	30829	28371	24.71	2523.80 (0.18%)	2	1.00
xibalba192	20808	19780	22.60	1609.27 (0.12%)	2	1.00
xibalba193	33376	30292	25.31	2760.36 (0.20%)	2	1.00
xibalba194	33045	29719	25.46	2723.77 (0.20%)	2	1.00
xibalba195	32442	29193	25.44	2673.83 (0.20%)	2	1.00
xibalba196	28748	24934	26.68	2394.60 (0.18%)	2	1.00
total	8226921	4670055	81.37	1367999.78 (100.00%)	362	751.48

The maximum capacity of the entire cluster is 157.97 Tcell/hour. Given that each cell requires 11 floating point operations (and not counting all the indexing/loading/storing operations) we have performed at least 15.048 Hexaflops or 2^63.71 flops.

If the processes were executed without parallelism, the computation would have taken 532.7 years.

The basic unit of work is called a cell, which is the computation of a table entry in the dynamic programming algorithm to align two sequences. The time is by and large proportional to the number of cells to be computed which is the product of the lengths of the sequences being aligned. Mcell stands for mega cells (1e6), Tcell for tera-cells (1e12).