You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-27 - 23:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Mon Apr 27, 2026 12:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20315236698,3598sleep14508-21kworker/1:009:18:131
16214219225,1904sleep212901-21kworker/2:210:46:402
13910279717,762sleep313927-21/usr/sbin/munin12:25:443
28865271710,693sleep028867-21sshd11:29:130
13307258196,139sleep113451-21gpgv10:40:371
21448249734,421sleep121513-21latency_hist07:40:471
15400241613,388sleep0191-21jbd2/mmcblk2p2-07:20:510
15400241613,388sleep0191-21jbd2/mmcblk2p2-07:20:510
2893241538,336sleep12964-21switchtime10:06:031
261642303104,182sleep126277-21unixbench_multi09:36:071
26874229114,113sleep10-21swapper/111:21:061
12164228216,247sleep012122-21latency_hist12:20:370
22785227489,125sleep10-21swapper/111:10:371
6897223256,139sleep16922-21awk08:36:011
13202222632,142sleep113247-21if_eth008:55:561
9330221510,153sleep19341-21fschecks_count12:10:531
24561221510,191sleep324549-21latency_hist07:50:473
24561221510,191sleep324549-21latency_hist07:50:473
128532215143,57sleep112915-21processes12:21:021
7959219872,110sleep10-21swapper/112:05:561
13705219836,88sleep10-21swapper/107:15:391
13705219836,88sleep10-21swapper/107:15:391
523321927,148sleep15379-21latency_hist10:15:371
7260218848,47sleep10-21swapper/110:20:581
3643218353,71sleep10-21swapper/111:51:061
21503218223,40sleep00-21swapper/009:21:070
21503218223,40sleep00-21swapper/009:21:070
26342217514,147sleep226375-21idleruntime11:20:552
5440217111,140sleep15503-21mailstats08:31:041
29383217111,116sleep129466-21cut11:30:561
11450217130,105sleep111551-21cut10:35:361
11450217130,105sleep111551-21cut10:35:361
32143216738,89sleep132243-21latency_hist11:40:371
31358216719,129sleep225884-21systemd-journal09:55:342
9042216510,140sleep19120-21mailstats10:26:031
28646216519,83sleep10-21swapper/109:45:561
21622216233,44sleep10-21swapper/107:40:511
1733215568,68sleep10-21swapper/111:45:581
28119215211,129sleep128133-21grep08:01:031
1495021489,23sleep10-21swapper/109:01:011
2798421429,97sleep128057-21sed11:25:591
18898214219,33sleep30-21swapper/307:31:043
9842214010,116sleep09921-21find12:11:040
5068214014,35sleep10-21swapper/111:56:031
24989213925,59sleep10-21swapper/107:51:011
13676213926,66sleep10-21swapper/110:40:451
13676213926,66sleep10-21swapper/110:40:451
186121387,67sleep10-21swapper/108:20:581
21654213773,23sleep10-21swapper/109:21:081
21654213773,23sleep10-21swapper/109:21:081
3374213320,92sleep32893-21apt-get08:25:453
29772213317,98sleep10-21swapper/108:06:041
2461621299,47sleep10-21swapper/111:15:501
24345212915,98sleep323860-21/usr/sbin/munin09:31:043
16517212814,69sleep116519-21cron10:50:341
32291212760,48sleep10-21swapper/108:15:371
15230212416,92sleep315357-21unixbench_multi09:01:073
157221230,7sleep20-21swapper/208:20:382
16234212116,88sleep316317-21cut09:05:553
14114212113,31sleep00-21swapper/007:15:580
14114212113,31sleep00-21swapper/007:15:580
330211935,71sleep1363-21kernelversion08:15:571
3570211623,77sleep03642-21switchtime11:51:050
2083211612,90sleep32094-21open_files08:21:043
10343211531,48sleep110378-21ps08:46:061
2898421145,11sleep00-21swapper/008:04:010
18925211435,43sleep118928-21cron09:15:351
13067211417,78sleep013215-21grep12:21:050
9186211127,60sleep19227-21proc_pri07:01:131
9186211127,60sleep19227-21proc_pri07:01:131
24280211014,73sleep024314-21/usr/sbin/munin09:31:010
23166211013,82sleep322310-21/usr/sbin/munin09:26:073
21425211014,23sleep30-21swapper/307:40:453
21240211014,77sleep10-21swapper/111:05:361
27557210918,50sleep127560-21cron08:00:341
2388421098,40sleep10-21swapper/109:30:491
21266210918,70sleep221411-21grep11:05:372
17439210922,17sleep10-21swapper/110:51:071
1081221097,68sleep110831-21apt-get12:15:451
23547210812,80sleep223560-21netstat07:46:092
23547210812,80sleep223560-21netstat07:46:092
7832210714,77sleep07852-21munin-run08:40:360
8816210614,17sleep30-21swapper/308:41:323
12329210610,77sleep012371-21/usr/sbin/munin10:36:080
17828210318,69sleep317880-21sort09:10:563
14816210316,67sleep214832-21/usr/sbin/munin09:00:582
14210210323,18sleep10-21swapper/112:25:591
27469210235,48sleep10-21swapper/109:41:031
2921210113,70sleep23013-21apt-config08:25:352
1774521019,79sleep217824-21fschecks_time09:10:552
16058210134,46sleep10-21swapper/112:31:031
708229821,60sleep27115-21grep10:20:552
2058529818,50sleep20-21swapper/209:20:372
2054329812,70sleep320641-21sed11:01:063
3138329711,70sleep031461-21switchtime11:36:030
696229619,59sleep36985-21ntp_states08:36:033
1561829613,61sleep315682-21cut12:30:533
3001129521,51sleep330117-21grep08:06:103
1057429532,46sleep10-21swapper/107:06:011
2373929410,35sleep10-21swapper/111:11:051
3095129324,44sleep330940-21apt-get08:10:543
2304329221,46sleep223163-21grep09:26:062
1885029113,37sleep10-21swapper/110:56:031
120542919,60sleep10-21swapper/108:51:071
3010229034,37sleep10-21swapper/109:50:491
1847129021,52sleep018550-21fschecks_time10:55:540
1707629015,62sleep017100-21kernelversion10:50:570
1390229013,59sleep10-21swapper/107:15:511
1390229013,59sleep10-21swapper/107:15:511
2379728931,21sleep123912-21timerandwakeup07:46:161
2379728931,21sleep123912-21timerandwakeup07:46:161
20312898,66sleep12081-21unin-run10:05:351
1776028820,51sleep10-21swapper/107:26:091
1776028820,51sleep10-21swapper/107:26:091
1763128817,55sleep016755-21/usr/sbin/munin07:26:080
1763128817,55sleep016755-21/usr/sbin/munin07:26:080
1877628719,33sleep10-21swapper/107:31:021
1537728718,53sleep10-21swapper/107:20:481
1537728718,53sleep10-21swapper/107:20:481
78262869,13sleep30-21swapper/312:05:543
614928624,16sleep10-21swapper/112:00:471
1369428620,40sleep213206-21apt-get10:40:462
1369428620,40sleep213206-21apt-get10:40:462
1046928612,60sleep010498-21nfs4_client10:30:590
1046928612,60sleep010498-21nfs4_client10:30:590
49428421,29sleep10-21swapper/111:41:051
466428417,33sleep00-21swapper/008:30:340
337728427,45sleep13400-21apt-get08:25:471
1735128413,57sleep217396-21grep07:26:022
1735128413,57sleep217396-21grep07:26:022
56562835,66sleep00-21swapper/011:57:560
2839528320,39sleep228538-21grep11:26:082
2430028321,43sleep224317-21sendmail11:15:342
2287728324,36sleep022898-21modprobe07:45:350
271428211,55sleep02295-21/usr/sbin/munin10:05:580
2330028211,50sleep023331-21latency11:10:560
553628113,49sleep10-21swapper/110:15:531
319492819,15sleep10-21swapper/109:55:581
315912810,7sleep20-21swapper/211:36:042
2785128113,51sleep227853-21sshd08:00:522
1683928128,23sleep30-21swapper/307:25:473
1683928128,23sleep30-21swapper/307:25:473
1235928114,53sleep212378-21fschecks_count07:11:032
1235928114,53sleep212378-21fschecks_count07:11:032
1188928120,43sleep311411-21apt-get10:35:463
1129228018,37sleep20-21swapper/212:16:022
435727916,49sleep34501-21latency_hist11:55:363
326782795,19sleep00-21swapper/009:57:100
1197427810,55sleep012053-21switchtime08:51:060
890627726,28sleep20-21swapper/207:01:062
890627726,28sleep20-21swapper/207:01:062
827827714,27sleep10-21swapper/108:40:411
620927718,43sleep36241-21awk08:35:343
413927710,14sleep20-21swapper/210:10:572
2529727719,35sleep225429-21tail11:16:072
297552760,7sleep20-21swapper/211:31:042
1932127618,42sleep319325-21diskstats09:15:533
1599227611,16sleep30-21swapper/310:46:203
145527618,41sleep21607-21unixbench_multi10:01:082
1431427612,49sleep214351-21ps07:16:042
1431427612,49sleep214351-21ps07:16:042
3253027517,42sleep332597-21df08:15:513
2572727517,27sleep00-21swapper/009:35:570
1580227520,38sleep20-21swapper/212:30:562
1077627516,42sleep310894-21sed10:31:063
1077627516,42sleep310894-21sed10:31:063
1076527517,31sleep20-21swapper/207:06:072
961127414,45sleep29642-21/usr/sbin/munin12:10:582
272122746,56sleep00-21swapper/007:56:130
2139327424,30sleep30-21swapper/309:21:063
2139327424,30sleep30-21swapper/309:21:063
2184627316,37sleep221459-21/usr/sbin/munin11:05:572
1633827322,34sleep10-21swapper/109:05:581
1118127318,32sleep011262-21latency_hist08:50:380
597727210,32sleep30-21swapper/310:16:033
241727225,25sleep30-21swapper/310:05:533
2151527211,17sleep20-21swapper/207:40:482
1608727215,40sleep216095-21/usr/sbin/munin09:05:462
692327124,26sleep20-21swapper/208:36:022
4212716,16sleep00-21swapper/011:41:040
2832027118,22sleep20-21swapper/209:45:372
2690327121,25sleep20-21swapper/207:56:072
1575627114,23sleep00-21swapper/010:46:170
1008827128,27sleep30-21swapper/307:05:433
1008827128,27sleep30-21swapper/307:05:433
278327015,23sleep00-21swapper/011:50:370
2747627011,34sleep00-21swapper/011:25:380
2709427013,46sleep027153-21fschecks_count09:40:540
1898627024,24sleep30-21swapper/310:56:053
1192327020,28sleep20-21swapper/210:35:532
828526920,31sleep27799-21apt-get08:40:462
2882126917,40sleep028839-21kernelversion09:45:590
223352697,17sleep10-21swapper/109:25:491
311726814,33sleep20-21swapper/211:50:552
1161026817,33sleep011581-21latency_hist07:10:370
2819026716,22sleep30-21swapper/311:26:053
2726826718,24sleep30-21swapper/309:40:573
2431926712,41sleep324377-21unin-run11:15:353
2075926714,30sleep00-21swapper/007:36:100
1420826722,26sleep30-21swapper/310:41:203
1420826722,26sleep30-21swapper/310:41:203
1364926717,32sleep013703-21grep07:15:380
1364926717,32sleep013703-21grep07:15:380
872526621,19sleep00-21swapper/007:01:000
872526621,19sleep00-21swapper/007:01:000
368626621,25sleep20-21swapper/208:26:042
2902426612,16sleep20-21swapper/209:46:052
2597226622,31sleep225995-21munin-run07:55:342
2105626615,35sleep221145-21/usr/sbin/munin09:20:572
2105626615,35sleep221145-21/usr/sbin/munin09:20:572
1911426612,33sleep019216-21/usr/sbin/munin07:31:130
1524326623,21sleep30-21swapper/307:20:383
1524326623,21sleep30-21swapper/307:20:383
138026613,38sleep00-21swapper/008:20:370
1289026621,26sleep30-21swapper/307:11:213
1289026621,26sleep30-21swapper/307:11:213
1265526620,24sleep30-21swapper/312:20:563
117926615,27sleep00-21swapper/010:01:020
641926522,19sleep20-21swapper/212:00:562
603326511,13sleep00-21swapper/012:00:380
2884526518,26sleep30-21swapper/308:02:143
2186026517,27sleep30-21swapper/311:05:583
1529126515,29sleep20-21swapper/212:30:362
1055626522,26sleep30-21swapper/312:15:353
623926416,28sleep36331-21perl12:00:533
569726420,19sleep00-21swapper/008:31:070
2593026418,26sleep00-21swapper/011:20:370
1845526412,35sleep218358-21/usr/sbin/munin10:55:522
114326415,31sleep31177-21/usr/sbin/munin10:01:013
711726314,20sleep30-21swapper/310:20:563
572026315,28sleep00-21swapper/010:15:560
2676626318,18sleep20-21swapper/209:40:362
1898326320,22sleep20-21swapper/207:31:112
1701726313,35sleep317024-21df_inode07:25:513
1701726313,35sleep317024-21df_inode07:25:513
1506326318,25sleep00-21swapper/012:28:520
2498326218,27sleep325014-21nfs_client11:16:003
880726114,26sleep20-21swapper/210:25:562
2174226115,21sleep30-21swapper/307:40:573
2011926111,24sleep00-21swapper/011:00:550
720626014,21sleep00-21swapper/010:20:570
37032605,11sleep30-21swapper/310:10:383
2488926020,19sleep30-21swapper/307:50:593
1817426014,26sleep318323-21sed10:55:373
486425919,19sleep30-21swapper/311:55:573
3095525923,21sleep00-21swapper/008:10:550
267002598,36sleep30-21swapper/311:21:043
1214225920,20sleep30-21swapper/312:20:363
854325818,18sleep30-21swapper/310:25:493
72525814,21sleep30-21swapper/311:41:073
190842586,11sleep00-21swapper/009:15:380
1631925818,16sleep00-21swapper/009:05:560
1600025814,25sleep00-21swapper/012:31:020
1354025820,20sleep20-21swapper/208:56:052
126002586,11sleep30-21swapper/308:54:373
994025718,22sleep20-21swapper/208:45:552
778425721,22sleep20-21swapper/212:05:532
3066925718,18sleep00-21swapper/009:51:040
28272576,37sleep00-21swapper/008:24:570
2599225712,22sleep30-21swapper/309:36:053
227192575,41sleep00-21swapper/011:10:150
2217625714,19sleep20-21swapper/207:41:082
670825615,17sleep20-21swapper/210:20:342
2641525612,18sleep30-21swapper/307:55:563
1580725620,19sleep20-21swapper/207:21:052
1580725620,19sleep20-21swapper/207:21:052
155825617,19sleep20-21swapper/211:45:552
146925613,19sleep30-21swapper/311:45:473
1054125613,27sleep210554-21ntp_states10:31:012
1054125613,27sleep210554-21ntp_states10:31:012
1045825610,16sleep30-21swapper/308:46:083
885025512,19sleep00-21swapper/008:41:330
50372556,38sleep00-21swapper/010:14:180
3244625515,19sleep20-21swapper/209:56:072
3235525519,20sleep20-21swapper/211:40:462
2926225512,25sleep029256-21apt-get11:30:510
2241225512,19sleep00-21swapper/009:25:520
1587025513,21sleep00-21swapper/009:05:360
1360825516,21sleep30-21swapper/308:56:063
3041025416,24sleep330411-21sshd11:34:133
2863025411,27sleep328636-21df_inode09:45:543
1048225417,21sleep30-21swapper/307:05:593
3209025316,21sleep332123-21munin-run11:40:353
301232538,30sleep030258-21unixbench_singl08:06:110
2994025312,24sleep230009-21swap08:06:092
231925311,25sleep22313-21apt-get10:05:482
891125215,19sleep00-21swapper/010:25:580
3113725115,16sleep20-21swapper/208:11:012
2385925118,20sleep20-21swapper/209:30:462
1395425114,22sleep013999-21irqstats10:41:120
1395425114,22sleep013999-21irqstats10:41:120
482825013,24sleep24862-21irqstats11:55:562
3058625015,17sleep30-21swapper/309:51:033
3186624913,18sleep30-21swapper/309:55:563
1275324917,17sleep20-21swapper/208:55:362
538124812,21sleep25406-21ntp_states08:31:012
2629024812,18sleep00-21swapper/007:55:490
1238524811,21sleep212379-21apt-get12:20:472
2361024711,22sleep023636-21ntp_states07:46:110
2361024711,22sleep023636-21ntp_states07:46:110
1788224712,16sleep00-21swapper/009:10:570
769724610,19sleep07693-21apt-get12:05:470
1813924613,15sleep20-21swapper/210:55:362
130042459,20sleep013067-21df08:55:510
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional