You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-12 - 16:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sun Jul 12, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
104752135312,1329sleep17002-21kworker/1:208:53:281
15723283327,791sleep215724-21apt-get09:09:042
771026835,667sleep226851-1kworker/2:2H08:44:202
5696258624,523sleep25739-21ntpq10:19:182
31022257614,545sleep131032-21/usr/sbin/munin11:39:091
32724255911,533sleep132756-21irqstats11:44:141
23772255628,485sleep226851-1kworker/2:2H09:34:282
13610247620,439sleep313669-21/usr/sbin/munin10:44:143
3333241511,192sleep23398-21switchtime08:29:222
1249223537,307sleep212566-21find07:14:232
7287235114,316sleep07373-21unin-run08:43:540
31223230694,194sleep231241-21modprobe09:58:512
14368229715,264sleep014446-21memory12:29:130
2400229349,141sleep20-21swapper/210:09:122
1006722538,207sleep210135-21fschecks_time10:34:122
18527224813,222sleep018528-21sshd09:18:390
18527224813,222sleep018528-21sshd09:18:390
24161223438,149sleep224204-21apt11:18:512
12429223044,116sleep212354-21/usr/sbin/munin08:59:102
9886221712,163sleep210000-21timerwakeupswit08:49:252
2102522169,193sleep221147-21timerandwakeup09:24:222
15999221232,144sleep216011-21netstat12:34:152
21323221011,186sleep121372-21grep07:44:101
24801220654,59sleep20-21swapper/207:54:212
23217220610,107sleep20-21swapper/209:33:512
29589220441,96sleep229673-21switchtime08:09:232
18118219811,167sleep00-21swapper/007:34:070
13102219528,19sleep20-21swapper/210:43:532
13880219427,128sleep214024-21latency_hist12:28:542
13643218028,133sleep30-21swapper/309:03:523
6272216713,104sleep25879-21/usr/sbin/munin08:39:152
5010216724,122sleep35157-21grep08:34:243
32758216610,142sleep2382-21cat11:44:152
1810621660,7sleep10-21swapper/110:58:531
14288216637,27sleep20-21swapper/212:29:112
18008216513,136sleep018104-21latency_hist10:58:530
3219421629,54sleep232197-21cron11:43:512
1631216227,95sleep21666-21ntpq08:24:172
13809215715,93sleep29729-21cyclictest10:44:212
2235215642,96sleep22323-21timerwakeupswit11:49:242
29631215049,88sleep229666-21latency11:34:142
17144214910,95sleep20-21swapper/210:54:272
856121478,85sleep20-21swapper/212:09:242
856121478,85sleep20-21swapper/212:09:242
2043214617,114sleep32075-21proc_pri11:49:213
11814214438,61sleep311782-21apt-get10:39:213
4969214044,48sleep20-21swapper/211:59:142
23443213827,87sleep123554-21timerwakeupswit07:49:231
13558213516,104sleep113561-21entropy07:19:101
17648213430,68sleep217680-21awk09:14:172
17648213430,68sleep217680-21awk09:14:172
15793213327,16sleep20-21swapper/207:24:242
22386213223,73sleep2364-21snmpd07:47:302
27549213027,55sleep20-21swapper/208:04:102
26340213029,83sleep20-21swapper/211:24:082
6527212921,65sleep20-21swapper/212:04:142
11525212625,82sleep20-21swapper/212:19:212
4263212524,17sleep20-21swapper/210:14:212
13710212415,94sleep113754-21ntpq10:44:161
2278212319,84sleep02304-21fschecks_count10:09:100
13929212317,23sleep20-21swapper/207:19:192
2812621205,12sleep00-21swapper/009:48:470
3096211918,85sleep33104-21/usr/sbin/munin11:54:023
2482211915,86sleep02494-21vmstat11:49:260
1836021188,64sleep20-21swapper/207:34:142
1043211718,77sleep20-21swapper/210:04:202
9616211615,78sleep30-21swapper/308:49:183
23556211419,20sleep29729-21cyclictest07:49:242
5446211319,10sleep20-21swapper/208:37:052
7789211225,76sleep2364-21snmpd10:24:312
5007211223,69sleep0278-21dbus-daemon10:18:510
3154321119,60sleep20-21swapper/209:59:072
17879211128,46sleep217944-21date07:33:532
25203210933,37sleep20-21swapper/211:19:272
38321088,86sleep332460-21/usr/sbin/munin11:44:163
21108210843,46sleep20-21swapper/211:08:532
21108210843,46sleep20-21swapper/211:08:532
15109210816,76sleep115122-21fschecks_count07:24:111
13913210621,45sleep20-21swapper/209:04:032
11257210611,49sleep20-21swapper/207:13:542
29664210529,20sleep20-21swapper/209:53:422
29664210529,20sleep20-21swapper/209:53:422
12623210514,74sleep012679-21fschecks_count12:24:120
10861210421,63sleep00-21swapper/007:09:360
3400210312,76sleep32758-21/usr/sbin/munin08:29:233
3150210320,22sleep00-21swapper/008:29:160
21349210223,42sleep221343-21apt-get11:09:052
21349210223,42sleep221343-21apt-get11:09:052
9610210132,47sleep09640-21munin-run07:08:510
25396210016,67sleep325405-21diskstats09:39:093
2029629912,73sleep320375-21wc07:39:243
869629812,67sleep08790-21memory10:29:170
18852988,53sleep00-21swapper/008:24:230
765729715,66sleep07193-21apt-get08:43:580
844429635,33sleep10-21swapper/107:04:131
1824729617,44sleep10-21swapper/110:59:071
3254429510,38sleep20-21swapper/208:19:172
2316629538,28sleep323196-21kernelversion11:14:153
189642959,38sleep20-21swapper/209:19:132
312129218,34sleep20-21swapper/211:54:052
2374729216,53sleep123736-21apt-get09:34:111
1823329219,34sleep20-21swapper/210:59:042
168412929,71sleep016911-21cat07:29:150
601329127,40sleep00-21swapper/012:03:520
2232629115,59sleep321934-21/usr/sbin/munin09:29:163
201082918,38sleep20-21swapper/207:39:182
968129021,54sleep20-21swapper/207:08:532
936928822,44sleep09388-21apt-get12:14:020
2722828827,46sleep227369-21sed08:03:542
153742886,48sleep21-21systemd10:49:222
2687128723,19sleep20-21swapper/209:44:052
2040528715,56sleep320482-21grep11:04:223
2040528715,56sleep320482-21grep11:04:223
100028622,30sleep00-21swapper/008:23:550
3140728522,18sleep20-21swapper/208:14:242
263832856,68sleep20-21swapper/209:41:232
3126728413,47sleep130535-21/usr/sbin/munin08:14:231
289792846,67sleep20-21swapper/211:32:402
1571728426,38sleep115209-21apt-get09:09:031
1394428414,50sleep014019-21find10:44:230
1224528425,39sleep3225-21systemd-journal12:23:503
1980428313,47sleep019853-21/usr/sbin/munin11:04:060
1980428313,47sleep019853-21/usr/sbin/munin11:04:060
331828012,54sleep13373-21/usr/sbin/munin11:54:121
2318128020,41sleep323215-21/usr/sbin/munin07:49:173
1969428011,20sleep00-21swapper/007:39:080
1667528017,29sleep316664-21apt-get10:54:083
1369528022,34sleep113667-21latency_hist09:03:541
123428024,34sleep31230-21apt-get08:24:063
862227922,42sleep28674-21if_eth010:29:152
316422790,9sleep00-21swapper/009:59:120
2541227926,28sleep10-21swapper/109:39:111
2336227917,45sleep123398-21wc11:14:211
1128927927,39sleep20-21swapper/208:54:222
2526427815,27sleep30-21swapper/311:19:283
1911827817,45sleep30-21swapper/307:35:363
1253527821,36sleep112674-21unixbench_singl10:39:401
123727813,18sleep10-21swapper/108:24:081
1089927819,45sleep111006-21grep08:54:121
2967527719,18sleep30-21swapper/308:09:243
1525627715,45sleep115285-21/usr/sbin/munin10:49:171
1454327723,31sleep114578-21grep12:29:181
1704127622,38sleep317141-21latency_hist09:13:533
1569927619,23sleep00-21swapper/009:09:010
2793927518,26sleep00-21swapper/008:04:200
1905127525,24sleep30-21swapper/310:59:253
846327413,45sleep18560-21timerandwakeup12:09:221
846327413,45sleep18560-21timerandwakeup12:09:221
1131727422,32sleep311394-21gpgv10:38:533
2818327323,27sleep30-21swapper/309:48:543
257752735,27sleep10-21swapper/111:21:161
1943927314,43sleep019508-21switchtime09:19:240
1878227311,43sleep018851-21/usr/sbin/munin10:59:220
1563027316,32sleep115624-21apt-get12:34:061
634927217,40sleep36445-21grep12:04:113
63452728,12sleep00-21swapper/010:20:490
3214127215,34sleep30-21swapper/308:19:053
2635327219,34sleep026428-21fschecks_time11:24:100
2105827214,43sleep121107-21systemctl11:08:521
2105827214,43sleep121107-21systemctl11:08:521
1384327221,27sleep10-21swapper/112:28:531
951127123,27sleep00-21swapper/008:49:150
352627122,30sleep33670-21sort10:13:543
2761427119,25sleep30-21swapper/311:28:533
2369927118,22sleep00-21swapper/007:49:260
122027117,34sleep31301-21switchtime10:04:253
939227022,32sleep20-21swapper/212:14:042
1696627015,23sleep30-21swapper/312:38:533
1550927019,28sleep30-21swapper/307:24:213
1434527016,39sleep314361-21kernelversion12:29:123
1143427017,22sleep00-21swapper/008:54:240
63362698,43sleep36337-21sshd10:20:443
2976426922,27sleep10-21swapper/111:34:171
382526813,24sleep30-21swapper/310:14:103
2559226822,24sleep00-21swapper/009:39:150
1420026813,42sleep114231-21latency09:04:141
1058526814,22sleep10-21swapper/110:34:231
1030426814,31sleep19913-21/usr/sbin/munin07:09:171
438226722,29sleep04424-21awk11:58:510
325102675,11sleep10-21swapper/108:19:161
3082926712,40sleep030916-21sed08:14:140
257352678,13sleep00-21swapper/007:58:550
190026719,31sleep11914-21sh10:08:511
695526616,28sleep00-21swapper/010:24:080
2872326615,33sleep328741-21munin-run08:08:513
2425326616,30sleep0362-21lldpd11:18:530
1944726619,24sleep10-21swapper/107:38:531
1141326625,18sleep00-21swapper/010:38:540
104152665,13sleep10-21swapper/112:15:411
469826522,28sleep34711-21sshd11:59:073
2172226523,26sleep30-21swapper/311:09:183
2172226523,26sleep30-21swapper/311:09:183
1279126520,27sleep10-21swapper/107:14:261
917726416,17sleep00-21swapper/012:13:540
917726416,17sleep00-21swapper/012:13:540
77932645,21sleep10-21swapper/110:24:341
568026410,26sleep10-21swapper/110:19:171
2836426315,30sleep128377-21awk08:04:251
76632629,25sleep10-21swapper/108:44:031
3061726220,21sleep30-21swapper/311:38:513
2842126217,25sleep30-21swapper/309:49:073
2842126217,25sleep30-21swapper/309:49:073
215526220,21sleep10-21swapper/111:49:231
1614326217,27sleep316192-21apt-config10:53:513
1305126213,30sleep013064-21idleruntime-cro10:43:510
1226026221,22sleep00-21swapper/007:14:170
644826115,24sleep00-21swapper/012:04:120
566626111,35sleep15791-21latency_hist08:38:541
319092615,13sleep00-21swapper/011:40:090
3047226118,22sleep00-21swapper/009:54:200
2617826116,19sleep30-21swapper/307:59:143
144592615,13sleep00-21swapper/007:20:220
819026012,36sleep38198-21ntp_states08:44:503
715026016,28sleep37151-21sendmail-mta08:43:413
2154526016,22sleep00-21swapper/011:09:150
2154526016,22sleep00-21swapper/011:09:150
210012606,40sleep10-21swapper/107:43:541
1698626016,28sleep317023-21ntp_states07:29:183
1146726019,22sleep10-21swapper/112:19:201
1119426017,28sleep311200-21modprobe07:13:513
890825916,29sleep38968-21/usr/sbin/munin10:29:243
852125916,28sleep18613-21fschecks_time10:29:141
812925922,20sleep30-21swapper/310:28:523
326625912,31sleep13332-21cat08:29:211
3162925917,23sleep30-21swapper/309:59:123
2643025912,19sleep10-21swapper/111:24:111
253162594,12sleep10-21swapper/107:55:211
2478725915,25sleep00-21swapper/011:19:090
1250925913,26sleep30-21swapper/312:24:033
41102587,11sleep10-21swapper/110:14:161
2947625815,26sleep00-21swapper/008:09:210
2374625811,23sleep00-21swapper/009:34:100
92125715,24sleep01014-21memory10:04:180
432725718,23sleep04323-21apt-get08:34:050
3093325716,24sleep330930-21apt-get11:39:033
3033525718,19sleep30-21swapper/309:54:153
2374425718,22sleep323231-21apt-get09:34:093
2334825715,18sleep00-21swapper/009:33:530
1895025716,26sleep118963-21fschecks_count09:19:121
160042576,39sleep00-21swapper/007:25:190
1316025716,24sleep313217-21modprobe07:18:523
1298625712,25sleep013088-21/usr/sbin/munin08:59:240
2265825616,19sleep10-21swapper/111:13:531
2265825616,19sleep10-21swapper/111:13:531
2077225618,23sleep320784-21grep09:24:153
1862425615,17sleep30-21swapper/309:18:543
1862425615,17sleep30-21swapper/309:18:543
1525825616,16sleep00-21swapper/009:08:530
1145225618,15sleep00-21swapper/012:19:190
792025513,26sleep37929-21sshd12:09:083
792025513,26sleep37929-21sshd12:09:083
74252554,13sleep10-21swapper/112:07:451
2781925513,24sleep027812-21apt-get11:29:070
590425415,19sleep10-21swapper/108:39:061
41325415,19sleep10-21swapper/110:03:541
2195925413,25sleep121953-21apt-get09:29:061
1411325415,27sleep314146-21idleruntime09:04:123
835125313,23sleep38347-21apt-get07:04:073
2879925315,20sleep10-21swapper/109:49:171
2879925315,20sleep10-21swapper/109:49:171
2687025313,25sleep126864-21apt-get09:44:041
2567025314,21sleep30-21swapper/307:58:543
2174825312,14sleep10-21swapper/109:28:531
1105925315,23sleep311072-21irqstats08:54:153
2441925212,23sleep024424-21/usr/sbin/munin07:54:110
1619525216,20sleep00-21swapper/010:53:520
1570725212,23sleep015714-21diskstats12:34:090
136425214,19sleep00-21swapper/011:48:540
2854325112,19sleep00-21swapper/009:49:120
2854325112,19sleep00-21swapper/009:49:120
2782025115,18sleep10-21swapper/111:29:081
259772517,18sleep10-21swapper/107:59:081
2163825118,17sleep30-21swapper/307:44:163
1979125113,22sleep319792-21sshd09:19:393
1736625114,20sleep10-21swapper/109:14:121
1736625114,20sleep10-21swapper/109:14:121
1654725114,22sleep116565-21apt-get07:29:041
1473525114,19sleep30-21swapper/307:23:533
218325012,22sleep12185-21apt-get10:09:031
492324913,20sleep14968-21irqstats11:59:131
3063424911,23sleep330641-21diskstats08:14:093
3023524912,25sleep129943-21/usr/sbin/munin09:54:131
2942724912,21sleep029438-21df_inode11:34:070
2040824914,22sleep020404-21apt-get09:24:040
985924810,22sleep39865-21ntp_states12:14:173
2908724813,20sleep129093-21df_inode08:09:101
2868324714,20sleep328721-21users11:29:263
2123724410,17sleep00-21swapper/007:44:050
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional