You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 03:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Fri Jul 10, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5829278916,758sleep05838-21/usr/sbin/munin21:14:400
5829278916,758sleep05838-21/usr/sbin/munin21:14:400
2454626076,589sleep124593-21ntpq23:59:281
2454626076,589sleep124593-21ntpq23:59:281
1419724676,447sleep114241-21ntpq19:59:271
20489246613,438sleep320495-21ntpq20:19:283
2194923927,372sleep121985-21ntpq20:24:261
511623790,18sleep231349-21kworker/u8:422:59:002
511623790,18sleep231349-21kworker/u8:422:59:002
10319236315,324sleep110466-21unixbench_singl21:29:351
23269232426,288sleep1364-21snmpd22:13:331
25404232217,273sleep10-21swapper/120:34:321
10864230012,274sleep310912-21unin-run21:34:013
2055229956,185sleep10-21swapper/122:49:021
958322989,275sleep09591-21ntpq19:44:280
29135225811,233sleep329145-21ntpq19:04:263
29135225811,233sleep329145-21ntpq19:04:263
20556223240,22sleep120631-21grep20:19:301
20503222947,69sleep120508-21cpu22:04:161
20503222947,69sleep120508-21cpu22:04:161
20212222216,185sleep120238-21munin-plugin-st22:04:001
20212222216,185sleep120238-21munin-plugin-st22:04:001
7059220213,175sleep37101-21fschecks_count23:04:233
7059220213,175sleep37101-21fschecks_count23:04:233
29588220128,161sleep1364-21snmpd20:49:211
8666219719,91sleep10-21swapper/121:24:351
1355721939,144sleep113496-21latency_hist19:59:031
9823219112,161sleep39854-21idleruntime-cro23:14:003
14543218416,153sleep314584-21ls21:44:233
5094218327,99sleep15098-21cron22:58:591
5094218327,99sleep15098-21cron22:58:591
3085521789,156sleep130893-21processes00:19:311
6190217621,100sleep11-21systemd21:19:011
6190217621,100sleep11-21systemd21:19:011
6618217348,67sleep16748-21timerwakeupswit19:34:321
3282217114,97sleep13314-21proc_pri00:34:291
31626216711,138sleep031645-21apt-get22:39:120
30281216744,87sleep130317-21cat22:34:201
18713216663,55sleep10-21swapper/121:59:031
17365216017,87sleep10-21swapper/120:09:281
11140215514,53sleep10-21swapper/119:49:301
1631921547,33sleep10-21swapper/123:34:131
1631921547,33sleep10-21swapper/123:34:131
29370215316,25sleep00-21swapper/000:14:330
29267215320,52sleep10-21swapper/122:29:361
23602215220,87sleep00-21swapper/022:14:200
23602215220,87sleep00-21swapper/022:14:200
15476215216,88sleep115626-21tail23:29:321
7945215146,50sleep10-21swapper/119:39:241
3603215144,62sleep13724-21latency_hist22:54:031
3603215144,62sleep13724-21latency_hist22:54:031
3066215124,22sleep30-21swapper/322:49:333
3066215124,22sleep30-21swapper/322:49:333
32277215016,57sleep10-21swapper/100:24:361
27904214718,118sleep328030-21sed00:09:323
771421466,17sleep10-21swapper/121:23:211
771421466,17sleep10-21swapper/121:23:211
1023421465,12sleep30-21swapper/323:14:243
29053213632,88sleep11-21systemd20:49:011
29053213632,88sleep11-21systemd20:49:011
14381213522,94sleep20-21swapper/219:59:322
4817213418,82sleep24829-21netstat19:29:242
1333621348,72sleep10-21swapper/121:39:331
18403213210,107sleep118460-21find23:39:301
18403213210,107sleep118460-21find23:39:301
1792221287,99sleep00-21swapper/023:39:160
1792221287,99sleep00-21swapper/023:39:160
2944021279,104sleep129570-21timerwakeupswit00:14:341
11672212737,71sleep10-21swapper/123:19:101
10841212319,80sleep00-21swapper/019:49:210
16808212246,56sleep10-21swapper/121:50:311
13516212210,95sleep10-21swapper/123:24:231
3775212027,41sleep13819-21ntpq21:09:241
2658211918,43sleep10-21swapper/119:24:021
21012211954,15sleep10-21swapper/123:49:151
28866211613,89sleep328873-21/usr/sbin/munin00:14:203
14272211612,71sleep10-21swapper/121:44:131
5099211516,80sleep35140-21/usr/sbin/munin21:14:273
5099211516,80sleep35140-21/usr/sbin/munin21:14:273
32480211541,24sleep10-21swapper/120:59:031
23598211515,32sleep323100-21/usr/sbin/munin20:29:293
2228211512,16sleep00-21swapper/019:19:330
1918211513,87sleep31919-21sshd22:47:553
27506211410,51sleep127536-21munin-run20:44:001
12956211418,77sleep212967-21sh23:24:002
27822211211,86sleep227903-21sed00:09:312
799211145,25sleep10-21swapper/120:59:311
2534321119,14sleep00-21swapper/022:19:050
21034211011,16sleep30-21swapper/322:04:303
21034211011,16sleep30-21swapper/322:04:303
3259821097,55sleep10-21swapper/119:14:251
18930210910,52sleep119012-21cut20:14:291
416210811,79sleep3465-21fw_packets20:59:213
542021068,45sleep15447-21grep22:59:201
13246210622,41sleep00-21swapper/023:24:140
32660210529,65sleep1364-21snmpd22:40:501
27557210516,72sleep327648-21latency_hist20:44:023
2648921045,12sleep10-21swapper/122:21:321
11462210323,56sleep10-21swapper/123:19:021
831821029,73sleep10-21swapper/123:09:011
831821029,73sleep10-21swapper/123:09:011
29113210229,24sleep229077-21munin-plugin-st20:49:022
29113210229,24sleep229077-21munin-plugin-st20:49:022
24239210114,33sleep324324-21perl23:59:223
24239210114,33sleep324324-21perl23:59:223
864021007,16sleep10-21swapper/123:09:171
37442986,14sleep00-21swapper/021:09:230
33852977,54sleep13445-21find19:24:301
189629715,55sleep10-21swapper/121:04:201
118242979,41sleep10-21swapper/121:34:321
3019529515,60sleep030278-21tune2fs22:34:190
2920229521,56sleep329405-21dpkg20:49:033
2920229521,56sleep329405-21dpkg20:49:033
2585829514,62sleep225400-21/usr/sbin/munin22:19:522
237002959,43sleep10-21swapper/122:14:261
237002959,43sleep10-21swapper/122:14:261
972729412,15sleep00-21swapper/021:29:220
290729431,37sleep02971-21fschecks_time00:34:200
244142946,12sleep20-21swapper/223:59:252
244142946,12sleep20-21swapper/223:59:252
10422948,19sleep00-21swapper/019:19:000
3269129318,60sleep232712-21dpkg20:59:122
231242939,41sleep10-21swapper/120:29:151
2287729312,67sleep022887-21unin-run20:29:020
48029222,54sleep3627-21unixbench_singl00:24:503
1479529212,60sleep014848-21cpuspeed23:29:150
112229218,23sleep30-21swapper/319:19:033
1008529225,48sleep20-21swapper/221:29:322
927129126,52sleep10-21swapper/119:44:211
346029124,53sleep33472-21fschecks_count21:09:183
344829116,57sleep22884-21/usr/sbin/munin19:24:312
3037229137,39sleep330461-21fschecks_time19:09:223
2616129113,55sleep225675-21/usr/sbin/munin00:04:302
1172629010,66sleep011823-21cat21:34:310
675028920,31sleep20-21swapper/219:34:332
3065628928,39sleep330719-21latency00:19:253
1811628917,45sleep018267-21unixbench_singl21:54:340
2253128829,11sleep10-21swapper/123:54:111
550528629,10sleep10-21swapper/119:31:211
2266028621,42sleep222778-21timerandwakeup22:09:302
2734828518,54sleep227359-21netstat22:24:322
2734828518,54sleep227359-21netstat22:24:322
1632828516,56sleep216407-21readlink21:49:312
160682858,54sleep10-21swapper/120:04:321
1596328519,49sleep016066-21grep20:04:310
2955928419,48sleep029067-21apt-get20:49:120
2390528415,50sleep00-21swapper/023:59:030
65152829,61sleep0364-21snmpd23:01:530
282972820,6sleep00-21swapper/022:29:010
282972820,6sleep00-21swapper/022:29:010
211962829,60sleep021255-21cat23:49:220
1995928221,48sleep10-21swapper/123:44:301
2793528018,11sleep10-21swapper/122:25:121
2793528018,11sleep10-21swapper/122:25:121
2721128019,11sleep10-21swapper/100:09:051
2077128022,34sleep00-21swapper/022:04:230
2077128022,34sleep00-21swapper/022:04:230
1852228019,42sleep318567-21cat23:39:323
1852228019,42sleep318567-21cat23:39:323
1526228015,28sleep20-21swapper/223:29:292
2096927910,56sleep221032-21ps22:04:292
2096927910,56sleep221032-21ps22:04:292
1955127919,39sleep319614-21fschecks_time23:44:203
1198027922,34sleep20-21swapper/219:54:032
304022780,6sleep20-21swapper/220:49:502
289792780,6sleep00-21swapper/019:04:220
289792780,6sleep00-21swapper/019:04:220
224927820,12sleep10-21swapper/100:30:251
820427722,28sleep20-21swapper/221:24:232
809427713,26sleep20-21swapper/219:39:292
571327722,37sleep31-21systemd19:34:013
2223127725,22sleep00-21swapper/022:09:200
756727617,34sleep20-21swapper/223:04:342
756727617,34sleep20-21swapper/223:04:342
384327618,18sleep00-21swapper/022:54:180
384327618,18sleep00-21swapper/022:54:180
2504327610,14sleep20-21swapper/220:34:252
148927640,23sleep0225-21systemd-journal21:03:300
2994927528,29sleep230003-21systemd-run19:09:012
2994927528,29sleep230003-21systemd-run19:09:012
299427529,24sleep00-21swapper/019:24:210
2666727519,12sleep10-21swapper/100:05:181
673227419,26sleep30-21swapper/323:04:023
3149327415,12sleep10-21swapper/119:13:281
1823227415,45sleep018379-21latency_hist20:14:040
1760027414,20sleep00-21swapper/023:39:000
1760027414,20sleep00-21swapper/023:39:000
175627418,17sleep20-21swapper/200:29:302
72272737,58sleep02548-21kworker/0:019:38:350
1194827316,27sleep20-21swapper/221:34:332
45292729,13sleep00-21swapper/019:29:180
337527210,30sleep20-21swapper/221:09:132
295027217,40sleep23065-21timerwakeupswit22:49:322
295027217,40sleep23065-21timerwakeupswit22:49:322
2596827216,42sleep326044-21systemctl22:20:013
3089927112,30sleep20-21swapper/200:19:322
256552718,15sleep00-21swapper/020:35:280
1296927113,44sleep313008-21unin-run23:24:013
2779627013,36sleep027846-21cpuspeed20:44:160
2779627013,36sleep027846-21cpuspeed20:44:160
250227018,21sleep00-21swapper/022:49:210
250227018,21sleep00-21swapper/022:49:210
1979327021,25sleep20-21swapper/220:19:022
1664927020,23sleep30-21swapper/320:09:023
1340627018,30sleep20-21swapper/221:39:342
135826920,27sleep00-21swapper/000:29:190
415526814,26sleep20-21swapper/200:39:022
3047726816,23sleep20-21swapper/222:34:252
1442526810,28sleep30-21swapper/319:59:333
814126713,38sleep08143-21diskstats21:24:210
3054926717,22sleep20-21swapper/219:09:252
256802675,13sleep30-21swapper/320:35:473
991726613,37sleep210071-21cut23:14:032
3006926616,30sleep330064-21apt-get22:34:143
2957226616,33sleep229696-21/usr/sbin/munin00:14:352
1901626610,32sleep319025-21df_inode21:59:173
1901626610,32sleep319025-21df_inode21:59:173
2694926517,34sleep326957-21acpi22:24:133
2694926517,34sleep326957-21acpi22:24:133
2071926514,33sleep220743-21latency_hist23:48:592
198922656,11sleep00-21swapper/021:59:590
198922656,11sleep00-21swapper/021:59:590
112202656,11sleep00-21swapper/023:16:550
1020326525,24sleep20-21swapper/223:14:232
3114026420,25sleep30-21swapper/320:54:163
162726415,20sleep20-21swapper/219:19:222
266762637,46sleep00-21swapper/000:05:250
1590926311,39sleep015988-21fschecks_time21:49:210
142192637,11sleep20-21swapper/223:25:212
1122126322,21sleep30-21swapper/321:34:183
3011526210,24sleep30-21swapper/320:49:483
1854126224,20sleep00-21swapper/020:14:200
138532625,16sleep00-21swapper/021:42:180
1225126219,24sleep00-21swapper/023:19:300
3027626113,21sleep00-21swapper/019:09:150
711926017,22sleep00-21swapper/023:04:240
711926017,22sleep00-21swapper/023:04:240
651026015,27sleep06504-21apt-get21:19:170
651026015,27sleep06504-21apt-get21:19:170
2077526019,21sleep00-21swapper/023:49:020
1682826017,22sleep20-21swapper/223:34:302
1682826017,22sleep20-21swapper/223:34:302
135926018,21sleep30-21swapper/319:19:153
972125921,18sleep30-21swapper/319:44:313
3220025914,26sleep232169-21apt-get00:24:332
3188325914,27sleep231981-21munin-node19:14:052
2489925912,22sleep00-21swapper/023:59:350
2489925912,22sleep00-21swapper/023:59:350
2383525920,18sleep20-21swapper/220:29:322
2366125913,27sleep023731-21sensors_temp20:29:300
22092598,14sleep00-21swapper/021:04:270
2101425919,25sleep321063-21/usr/sbin/munin23:49:173
1553725916,17sleep20-21swapper/220:04:212
1236125917,25sleep212428-21sort19:54:192
766325813,30sleep07699-21fschecks_count19:39:190
626225817,17sleep20-21swapper/221:19:022
626225817,17sleep20-21swapper/221:19:022
3215725812,23sleep00-21swapper/000:24:140
1626925819,25sleep316276-21ntp_states21:49:283
1043325817,21sleep20-21swapper/219:49:022
336025714,24sleep33366-21ntp_states19:24:283
1937725714,18sleep20-21swapper/221:59:282
1937725714,18sleep20-21swapper/221:59:282
1889725717,23sleep318903-21ntp_states20:14:273
1741225716,24sleep217499-21mailstats20:09:292
170025717,24sleep31724-21ntp_states00:29:263
1342925712,26sleep30-21swapper/323:24:213
112752578,11sleep30-21swapper/319:49:323
76552565,15sleep30-21swapper/321:22:353
76552565,15sleep30-21swapper/321:22:353
2341525612,17sleep30-21swapper/322:14:033
2341525612,17sleep30-21swapper/322:14:033
231362568,38sleep30-21swapper/322:11:493
2139825613,18sleep20-21swapper/220:24:032
19042566,11sleep20-21swapper/221:04:212
1734525617,18sleep30-21swapper/320:09:263
1553225615,28sleep015552-21modprobe21:49:010
1379425615,25sleep013790-21apt-get19:59:140
1343325617,21sleep30-21swapper/319:58:283
1221425614,25sleep012208-21apt-get19:54:130
554125514,22sleep20-21swapper/222:59:252
50325517,19sleep20-21swapper/222:44:022
3105625518,17sleep00-21swapper/000:19:340
2866625514,17sleep10-21swapper/119:04:121
2866625514,17sleep10-21swapper/119:04:121
2770925513,25sleep027743-21ps00:09:290
1479325514,25sleep314788-21apt-get23:29:143
82025412,25sleep2824-21df_inode22:44:192
688625418,22sleep20-21swapper/221:19:272
688625418,22sleep20-21swapper/221:19:272
2983225419,19sleep00-21swapper/022:34:030
2939925414,20sleep20-21swapper/222:29:372
2860525415,20sleep30-21swapper/322:29:223
863525313,17sleep00-21swapper/023:09:160
3245225312,24sleep032479-21munin-run20:59:020
262725314,15sleep30-21swapper/300:34:033
2157825316,18sleep30-21swapper/320:24:163
1745325314,23sleep217457-21diskstats21:54:202
1734725316,21sleep317366-21apt-get21:54:123
757625213,23sleep37572-21apt-get19:39:143
599725214,23sleep35994-21apt-get19:34:133
2646525212,18sleep20-21swapper/220:39:212
194025215,19sleep32024-21perl21:04:223
1558425212,18sleep20-21swapper/221:49:032
517625114,22sleep30-21swapper/322:59:023
517625114,22sleep30-21swapper/322:59:023
2657525113,17sleep00-21swapper/020:39:230
2655425112,23sleep326601-21users00:04:353
2367325112,16sleep20-21swapper/222:14:222
2367325112,16sleep20-21swapper/222:14:222
2253325113,24sleep322553-21apt-get23:54:133
1770325111,18sleep30-21swapper/323:39:033
1770325111,18sleep30-21swapper/323:39:033
931525011,19sleep39346-21munin-run21:29:013
2002925012,22sleep220023-21apt-get20:19:142
1403725014,19sleep30-21swapper/321:44:033
3208224711,21sleep332105-21ntp_states22:39:273
2162624712,17sleep20-21swapper/223:49:322
3113924610,19sleep20-21swapper/220:54:152
2381324615,16sleep20-21swapper/223:59:012
83324512,14sleep00-21swapper/022:44:210
2129024511,21sleep021337-21systemd-run20:24:010
1816124313,15sleep20-21swapper/223:39:222
1816124313,15sleep20-21swapper/223:39:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional