You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-15 - 07:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Fri May 15, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20091210848,1017sleep320132-21unin-run00:29:243
107432101850,910sleep310760-21unin-run23:59:243
13202496133,288sleep31353-21fschecks_count20:04:453
1516922789,255sleep315231-21mailstats22:29:523
1516922789,255sleep315231-21mailstats22:29:523
79692272166,40sleep38051-21gpgv22:09:253
95282265173,74sleep30-21swapper/320:29:483
17580224648,185sleep317656-21sed22:39:253
15759223718,197sleep015853-21fschecks_time00:14:430
405122349,210sleep34136-21cat23:39:253
22223223398,65sleep30-21swapper/300:34:453
17311223111,206sleep217403-21load20:54:462
425222424,117sleep3564-21unixbench_singl20:00:173
17781222210,189sleep317931-21grep20:54:553
16063221315,183sleep216153-21cat20:49:542
16063221315,183sleep216153-21cat20:49:542
12844221312,179sleep312847-21cron22:24:233
12844221312,179sleep312847-21cron22:24:233
22466220935,57sleep30-21swapper/322:54:393
22466220935,57sleep30-21swapper/322:54:393
31939220410,176sleep331309-21/usr/sbin/munin21:39:543
26910220112,173sleep326997-21latency_hist23:09:253
2980121997,181sleep3363-21snmpd19:51:323
20520219813,173sleep320615-21latency_hist19:24:273
7547218325,87sleep37745-21grep23:49:273
31655218116,151sleep131741-21apt-config19:59:261
1575521780,30sleep215759-21fschecks_count19:09:432
30439217212,145sleep330596-21grep21:34:573
18704217023,85sleep318794-21fschecks_time20:59:443
30398216812,142sleep330465-21fschecks_time19:54:433
15850216658,24sleep30-21swapper/319:09:453
1117621599,138sleep311201-21awk20:34:493
1117621599,138sleep311201-21awk20:34:493
9346215619,115sleep09412-21fschecks_time20:29:450
20996215267,20sleep30-21swapper/300:29:573
12770215216,118sleep112538-21/usr/sbin/munin00:04:471
2399121479,81sleep30-21swapper/321:14:543
17225214111,66sleep30-21swapper/300:19:353
14141213914,112sleep00-21swapper/019:04:420
23288213318,94sleep30-21swapper/321:14:263
57221319,109sleep3641-21basename21:44:463
18226213124,93sleep318314-21timerandwakeup19:14:543
27239213016,82sleep30-21swapper/323:09:463
16242212614,82sleep20-21swapper/222:34:402
1170212643,69sleep31162-21apt-get23:29:403
8362212520,88sleep18483-21timerwakeupswit20:24:531
25667212425,20sleep30-21swapper/321:19:553
20674212421,83sleep120687-21fschecks_count00:29:431
5193212015,82sleep20-21swapper/221:59:432
8640211919,77sleep30-21swapper/323:49:593
21953211820,79sleep20-21swapper/221:09:362
16814211729,23sleep30-21swapper/320:54:273
16814211729,23sleep30-21swapper/320:54:273
20894211631,44sleep00-21swapper/022:49:360
20894211631,44sleep00-21swapper/022:49:360
30800211534,13sleep30-21swapper/323:19:473
1468221139,18sleep30-21swapper/300:09:543
25672211220,73sleep225691-21cpu19:39:402
19171211123,70sleep019210-21ntpq00:24:490
18366211116,78sleep318439-21switchtime22:39:543
13694211068,21sleep30-21swapper/320:44:263
13694211068,21sleep30-21swapper/320:44:263
24313210932,14sleep30-21swapper/322:59:483
21525210940,14sleep30-21swapper/319:24:553
8455210829,14sleep30-21swapper/322:09:413
32161210725,26sleep332252-21fschecks_time23:24:443
22228210715,76sleep222325-21latency_hist22:54:262
22228210715,76sleep222325-21latency_hist22:54:262
1283421069,19sleep30-21swapper/300:04:493
5509210516,73sleep05555-21ntp_offset21:59:490
17043210520,63sleep30-21swapper/300:19:273
19099210417,69sleep018593-21/usr/sbin/munin20:59:530
30075210315,66sleep230162-21latency_hist19:54:262
556921027,81sleep35608-21ps21:59:523
4604210211,66sleep30-21swapper/320:14:463
1008221029,17sleep30-21swapper/322:14:423
1008221029,17sleep30-21swapper/322:14:423
6756210114,72sleep26862-21grep22:04:442
1206721015,13sleep30-21swapper/300:01:503
4422210024,61sleep04484-21grep20:14:430
12128210014,70sleep012204-21expr22:19:520
12128210014,70sleep012204-21expr22:19:520
340529922,52sleep23561-21unixbench_multi23:34:562
285302988,39sleep328529-21munin-run19:49:263
2674629814,61sleep30-21swapper/321:24:433
96629713,68sleep01038-21swap21:44:550
696529763,16sleep30-21swapper/320:20:153
3209829625,54sleep231617-21apt-get19:59:352
2586829523,58sleep025888-21kernelversion23:04:460
205462955,34sleep30-21swapper/322:48:223
733929426,52sleep07374-21basename23:49:240
1541629419,56sleep215532-21sed22:29:552
1541629419,56sleep215532-21sed22:29:552
569129317,58sleep30-21swapper/323:44:263
486829330,46sleep14992-21latency_hist21:59:261
428929311,17sleep00-21swapper/023:39:390
2968229319,52sleep029823-21grep23:14:560
256129214,19sleep30-21swapper/321:49:553
181429236,33sleep00-21swapper/023:29:580
478729112,20sleep30-21swapper/321:59:243
252432915,16sleep30-21swapper/319:37:593
1067429117,58sleep210742-21mandb23:59:232
2417929017,48sleep124194-21fschecks_count19:34:421
2005929023,48sleep220088-21munin-run00:29:232
97112898,17sleep20-21swapper/222:14:252
301432895,13sleep30-21swapper/323:18:343
2154428910,21sleep321674-21sed22:49:553
2154428910,21sleep321674-21sed22:49:553
1457228923,49sleep314669-21sensors_temp19:04:533
143112899,67sleep314352-21ntpq20:44:503
143112899,67sleep314352-21ntpq20:44:503
2120728812,50sleep30-21swapper/321:06:023
41628714,29sleep2419-21apt-get21:44:372
131302868,15sleep30-21swapper/322:24:353
131302868,15sleep30-21swapper/322:24:353
98728512,59sleep01006-21latency_hist20:04:250
287028520,40sleep02935-21fschecks_time20:09:420
287028520,40sleep02935-21fschecks_time20:09:420
2271328526,43sleep222798-21switchtime00:34:572
2068528513,57sleep020738-21/usr/sbin/munin21:05:510
2030428514,57sleep120306-21sshd22:45:171
683828418,45sleep26963-21grep20:20:142
216682845,15sleep10-21swapper/119:24:571
1808828414,54sleep118152-21ps19:14:521
1220528415,53sleep112334-21timerwakeupswit22:19:531
1220528415,53sleep112334-21timerwakeupswit22:19:531
1126828416,48sleep011329-21sendmail-msp22:19:230
1126828416,48sleep011329-21sendmail-msp22:19:230
397128311,59sleep2363-21snmpd20:13:282
397128311,59sleep2363-21snmpd20:13:282
1233528315,52sleep212476-21uptime22:19:542
1233528315,52sleep212476-21uptime22:19:542
1415428229,22sleep20-21swapper/219:04:442
1048328224,29sleep10-21swapper/122:14:521
1048328224,29sleep10-21swapper/122:14:521
3204328023,22sleep00-21swapper/021:39:550
2230528018,30sleep00-21swapper/000:34:460
100642800,7sleep00-21swapper/023:54:560
332027915,43sleep13403-21grep23:34:551
3002527914,45sleep0225-21systemd-journal19:54:240
284942798,57sleep30-21swapper/321:29:483
248892790,6sleep20-21swapper/219:34:552
2846127819,46sleep228552-21apt-key23:14:252
1761927819,29sleep00-21swapper/019:14:390
1431627815,45sleep214366-21/usr/sbin/munin00:09:452
267042775,61sleep30-21swapper/319:41:313
1476427715,28sleep00-21swapper/000:09:550
1098627614,25sleep00-21swapper/023:59:330
2104527513,40sleep121108-21/usr/sbin/munin21:06:001
2058527512,30sleep20-21swapper/221:05:492
2014127524,29sleep219913-21latency_hist21:04:272
1321727514,39sleep013235-21fschecks_count22:24:400
1321727514,39sleep013235-21fschecks_count22:24:400
1214527515,43sleep012293-21latency_hist20:39:260
1214527515,43sleep012293-21latency_hist20:39:260
2392627318,35sleep023962-21ntp_kernel_pll_21:14:510
1780227317,34sleep10-21swapper/100:19:531
1600327315,42sleep016149-21latency_hist22:34:260
1600327315,42sleep016149-21latency_hist22:34:260
1588127312,45sleep115919-21/usr/sbin/munin20:49:491
1588127312,45sleep115919-21/usr/sbin/munin20:49:491
1140127315,43sleep111444-21ntpq23:59:511
866627215,25sleep10-21swapper/122:09:521
2380627211,46sleep323827-21idleruntime-cro19:34:243
86872705,10sleep30-21swapper/320:25:483
654727024,26sleep20-21swapper/223:44:582
568127025,28sleep25729-21munin-run20:19:252
360927021,31sleep20-21swapper/221:54:422
2770227020,26sleep20-21swapper/219:44:532
2685927016,22sleep10-21swapper/123:09:231
2263327021,25sleep10-21swapper/100:34:561
920526923,19sleep10-21swapper/123:54:251
806526922,20sleep00-21swapper/022:09:260
676126916,29sleep16836-21grep20:20:131
32362698,45sleep30-21swapper/320:09:493
32362698,45sleep30-21swapper/320:09:493
3002826922,18sleep20-21swapper/221:34:482
283932690,10sleep00-21swapper/021:29:440
1226826918,32sleep20-21swapper/200:04:242
817026817,33sleep28198-21proc_pri20:24:502
3029226813,25sleep10-21swapper/121:34:551
3261726715,26sleep10-21swapper/120:00:101
3215726722,24sleep20-21swapper/223:24:432
3098926717,36sleep031022-21proc_pri23:19:520
2532226710,11sleep20-21swapper/221:19:462
1916226711,34sleep119229-21swap20:59:541
1134726716,26sleep20-21swapper/222:19:262
1134726716,26sleep20-21swapper/222:19:262
972726613,12sleep10-21swapper/120:29:541
2912226611,22sleep10-21swapper/123:14:441
2594326621,24sleep10-21swapper/119:39:471
2201626615,36sleep022076-21apt-config19:29:240
292626516,28sleep02984-21/usr/sbin/munin23:34:460
2308426520,28sleep223207-21/usr/sbin/munin22:54:552
2308426520,28sleep223207-21/usr/sbin/munin22:54:552
1572226517,25sleep10-21swapper/100:14:401
3028826415,35sleep230364-21sed23:19:262
2293326421,23sleep20-21swapper/219:30:032
2096826413,27sleep20-21swapper/200:29:562
1802326416,24sleep20-21swapper/219:14:492
1773626420,19sleep00-21swapper/000:19:520
1439526413,23sleep20-21swapper/220:44:532
1439526413,23sleep20-21swapper/220:44:532
3182826322,26sleep10-21swapper/123:24:251
3156226319,23sleep20-21swapper/221:39:442
270902637,20sleep20-21swapper/221:24:502
2385826318,25sleep10-21swapper/119:34:261
2237826313,29sleep022030-21apt-get19:29:280
1790526316,30sleep017918-21fschecks_count22:39:430
163326319,20sleep10-21swapper/123:29:551
3174926214,32sleep231961-21apt-config19:59:272
2847826214,26sleep128504-21modprobe19:49:241
268392629,43sleep00-21swapper/019:43:130
1781726219,27sleep217814-21apt-get22:39:362
1349926217,27sleep113527-21/usr/sbin/munin22:24:461
1349926217,27sleep113527-21/usr/sbin/munin22:24:461
1333226223,20sleep00-21swapper/000:04:580
37332617,15sleep00-21swapper/021:54:440
3186326119,19sleep20-21swapper/223:24:262
2749026113,32sleep027582-21/usr/sbin/munin19:44:470
2606726111,21sleep20-21swapper/223:04:522
249326114,28sleep22529-21awk20:09:242
2425226117,23sleep10-21swapper/122:59:461
234142618,11sleep00-21swapper/022:55:250
234142618,11sleep00-21swapper/022:55:250
1697326113,29sleep017075-21munin-node19:14:260
1448826117,24sleep00-21swapper/020:44:550
1448826117,24sleep00-21swapper/020:44:550
1310226121,23sleep10-21swapper/120:39:551
1310226121,23sleep10-21swapper/120:39:551
826126010,14sleep00-21swapper/023:49:520
615426012,34sleep05612-21kworker/u8:320:19:350
4252609,28sleep10-21swapper/121:44:401
1978626020,17sleep10-21swapper/119:19:561
1286126011,30sleep212923-21wc20:39:522
1286126011,30sleep212923-21wc20:39:522
3231025918,23sleep10-21swapper/123:24:461
2768825910,24sleep10-21swapper/123:09:561
2520125917,21sleep00-21swapper/021:19:440
2404325918,26sleep224039-21apt-get22:59:392
2055725916,24sleep00-21swapper/000:29:330
1902325911,29sleep119068-21irqstats00:24:461
1749925921,22sleep10-21swapper/122:39:231
1701225919,24sleep20-21swapper/200:19:262
2238125815,21sleep00-21swapper/021:09:500
465925721,19sleep10-21swapper/123:39:481
2140325713,28sleep10-21swapper/122:49:531
2140325713,28sleep10-21swapper/122:49:531
1283925715,26sleep012845-21ntp_states20:39:500
1283925715,26sleep012845-21ntp_states20:39:500
1076525721,15sleep10-21swapper/123:59:261
795125614,25sleep07980-21irqstats20:24:440
2861025620,20sleep10-21swapper/121:29:501
242525620,21sleep22447-21modprobe23:34:242
197925613,26sleep01973-21apt-get21:49:400
195725612,27sleep02091-21unixbench_singl20:04:580
1565725615,21sleep10-21swapper/119:09:391
783625516,24sleep27354-21apt-get23:49:342
428625515,24sleep24283-21apt-get23:39:372
2877025512,19sleep10-21swapper/119:49:401
2847925518,19sleep20-21swapper/221:29:472
2419225514,19sleep20-21swapper/221:14:572
2409325512,26sleep024088-21apt-get19:34:370
171225517,19sleep10-21swapper/120:04:551
142142557,36sleep20-21swapper/222:27:092
142142557,36sleep20-21swapper/222:27:092
1108125513,23sleep211175-21memory20:34:482
1108125513,23sleep211175-21memory20:34:482
732325416,17sleep10-21swapper/122:04:551
608225414,24sleep16070-21apt-get23:44:401
2536025420,19sleep00-21swapper/023:04:260
2102725417,16sleep20-21swapper/219:24:462
1703725412,25sleep117032-21apt-get20:54:371
152212547,10sleep10-21swapper/100:12:301
278625313,22sleep12853-21df20:09:391
278625313,22sleep12853-21df20:09:391
2732525313,20sleep00-21swapper/023:09:470
2352725312,24sleep123522-21apt-get21:14:401
2242725313,19sleep10-21swapper/121:09:511
198025314,23sleep12048-21df21:49:411
1549225312,15sleep00-21swapper/020:49:400
1549225312,15sleep00-21swapper/020:49:400
1903125211,16sleep00-21swapper/019:19:270
1256025215,22sleep212556-21apt-get00:04:402
1080825212,24sleep110803-21apt-get20:34:381
1080825212,24sleep110803-21apt-get20:34:381
892025110,25sleep08960-21ps22:09:590
518925114,22sleep15236-21users20:14:561
3060025114,15sleep00-21swapper/021:34:580
2666325113,21sleep026730-21df21:24:400
1966725111,14sleep20-21swapper/219:19:532
1792225113,20sleep118026-21perl22:39:441
47025012,20sleep0517-21users23:24:570
1937125010,15sleep20-21swapper/222:44:412
1793525017,14sleep00-21swapper/020:54:560
1529725013,21sleep115415-21sed22:29:541
1529725013,21sleep115415-21sed22:29:541
1476825015,20sleep114926-21sed19:04:551
1457225012,15sleep10-21swapper/120:44:561
1457225012,15sleep10-21swapper/120:44:561
933024913,22sleep29339-21df_inode20:29:432
2246424916,17sleep10-21swapper/122:54:371
2246424916,17sleep10-21swapper/122:54:371
510524811,23sleep15099-21apt-get21:59:381
1752324817,16sleep20-21swapper/200:19:462
2511624510,18sleep10-21swapper/121:19:411
321822447,13sleep10-21swapper/121:39:561
3157224410,17sleep031649-21systemd-run19:59:250
279672449,13sleep10-21swapper/121:29:261
1936924411,16sleep00-21swapper/022:44:390
20492437,14sleep20-21swapper/221:49:422
162402389,14sleep00-21swapper/022:34:380
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional