You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-20 - 19:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Apr 20, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
288972110977,11078sleep128702-21kworker/1:110:55:121
249622695222,6917sleep313613-21kworker/3:207:20:393
277662305410,3031sleep016684-21kworker/0:112:34:130
293902152517,1491sleep229343-21/usr/sbin/munin09:16:012
185372118022,1142sleep08988-21kworker/u8:008:41:190
154929996,981sleep132599-21kworker/1:009:28:121
2179429297,907sleep121877-21memory07:11:051
117472281128,25sleep10-21swapper/108:21:011
42332278131,34sleep14289-21latency07:56:221
27517225860,180sleep125884-21systemd-journal09:10:591
13051225769,171sleep112938-21/usr/sbin/munin10:06:191
13333222925,187sleep213361-21/usr/sbin/munin10:06:242
6247222614,200sleep36315-21grep11:26:183
6247222614,200sleep36315-21grep11:26:183
9685221450,140sleep10-21swapper/111:36:251
9685221450,140sleep10-21swapper/111:36:251
1564822087,190sleep113441-21snmpd10:14:141
29641219817,30sleep10-21swapper/107:35:591
1484821940,105sleep014853-21latency10:11:220
16022219114,160sleep115665-21/usr/sbin/munin11:56:231
2208321857,163sleep122161-21grep12:16:171
2407321789,155sleep124116-21gpgv10:40:591
8322217711,152sleep18347-21munin-run08:11:001
24695217517,137sleep10-21swapper/110:41:241
1420921727,153sleep114340-21cat08:26:301
253592170124,32sleep125488-21timerwakeupswit09:01:311
253592170124,32sleep125488-21timerwakeupswit09:01:311
3149216945,74sleep13185-21expr11:16:191
3149216945,74sleep13185-21expr11:16:191
214832167126,24sleep121560-21mailstats10:31:301
9857216513,128sleep10-21swapper/109:56:141
19039216315,72sleep119181-21gpgv10:26:011
20220216114,25sleep10-21swapper/108:46:211
2788421580,36sleep027888-21fschecks_time10:51:180
1873221577,35sleep10-21swapper/108:41:241
736821508,89sleep113441-21snmpd11:30:191
736821508,89sleep113441-21snmpd11:30:191
17463215012,116sleep317609-21latency_hist10:21:033
11202214923,110sleep00-21swapper/011:41:240
11202214923,110sleep00-21swapper/011:41:240
15426214811,18sleep10-21swapper/111:55:591
15426214811,18sleep10-21swapper/111:55:591
30913214715,17sleep10-21swapper/111:01:171
31557214514,112sleep131640-21tune2fs09:21:331
20761214514,114sleep00-21swapper/008:46:320
13297214411,53sleep10-21swapper/108:26:021
30672214211,54sleep10-21swapper/107:36:361
1179621418,64sleep10-21swapper/110:01:261
920721405,12sleep10-21swapper/109:51:411
15659213917,60sleep10-21swapper/108:31:311
26596213610,67sleep126741-21latency_hist07:26:021
526121349,109sleep10-21swapper/109:41:201
2745321349,12sleep10-21swapper/110:51:001
2759021319,107sleep10-21swapper/112:32:001
461021305,11sleep10-21swapper/109:37:201
3259421306,105sleep10-21swapper/111:06:221
23633213020,89sleep223680-21fschecks_count12:21:192
3010021299,108sleep130161-21processes09:16:291
3606212812,103sleep03625-21apt09:36:100
3041821285,10sleep10-21swapper/110:59:201
2680621285,11sleep10-21swapper/107:26:101
28385212434,62sleep328374-21apt-get07:31:173
19601212310,52sleep10-21swapper/110:27:141
1303521238,100sleep113166-21cat11:46:301
2332421218,56sleep123390-21if_eth008:56:181
2332421218,56sleep123390-21if_eth008:56:181
15061212015,89sleep215077-21awk08:31:122
21156211946,62sleep10-21swapper/107:07:091
17702211829,76sleep117746-21awk12:01:241
17702211829,76sleep117746-21awk12:01:241
580621165,99sleep10-21swapper/111:25:111
2091721149,50sleep10-21swapper/112:11:291
2608121136,96sleep10-21swapper/112:27:281
24716211215,80sleep224816-21perl10:41:252
18418211217,77sleep317933-21apt-get08:41:103
1666621127,45sleep10-21swapper/108:36:151
25933211126,66sleep226059-21wc07:21:312
16495211114,82sleep216645-21uptime11:56:322
20665210716,76sleep020707-21cut07:06:270
30601210614,76sleep230752-21latency_hist11:01:022
6118210540,12sleep10-21swapper/108:01:311
310521055,76sleep10-21swapper/109:33:161
23255210518,72sleep223275-21munin-plugin-st12:20:592
7202210310,36sleep17267-21awk09:46:291
4364210322,61sleep34484-21grep09:36:333
31984210216,61sleep20-21swapper/207:41:212
1479329922,57sleep214821-21modprobe08:30:592
1054129914,69sleep310585-21systemd-run11:40:593
1054129914,69sleep310585-21systemd-run11:40:593
3151529825,52sleep331655-21tail11:01:323
2482329732,53sleep10-21swapper/107:18:521
3058029611,69sleep230619-21ntpq07:36:332
2213629611,30sleep122215-21switchtime08:51:291
1149329510,69sleep211501-21df10:01:192
3007729311,68sleep330091-21ntpq09:16:273
1090229314,57sleep310413-21/usr/sbin/munin08:16:273
1656929239,30sleep30-21swapper/310:16:303
1123529233,44sleep10-21swapper/111:41:251
1123529233,44sleep10-21swapper/111:41:251
884629114,62sleep28912-21switchtime09:51:302
353029131,27sleep10-21swapper/107:54:331
353029131,27sleep10-21swapper/107:54:331
2509429025,32sleep00-21swapper/012:26:140
1720929031,11sleep10-21swapper/110:19:031
951528950,19sleep10-21swapper/108:12:131
51528928,10sleep10-21swapper/107:45:401
203028912,61sleep325884-21systemd-journal09:31:123
201628931,10sleep10-21swapper/107:50:001
201628931,10sleep10-21swapper/107:50:001
1887428923,53sleep318889-21kernelversion07:01:223
258628827,10sleep10-21swapper/111:13:571
2380928848,23sleep10-21swapper/112:21:221
89462875,70sleep10-21swapper/111:35:441
2904128728,10sleep10-21swapper/109:12:371
1789128716,52sleep217980-21threads12:01:292
1789128716,52sleep217980-21threads12:01:292
327532866,12sleep00-21swapper/011:06:250
2456828627,36sleep324586-21apt10:41:093
156928618,51sleep31716-21sed07:46:313
156928618,51sleep31716-21sed07:46:313
2971728518,45sleep029755-21fschecks_count09:16:190
1981528531,28sleep10-21swapper/112:07:091
1908128518,42sleep118587-21/usr/sbin/munin07:01:281
396628217,49sleep34036-21df07:56:163
312992800,6sleep00-21swapper/011:01:280
2939328022,29sleep30-21swapper/309:16:023
2349528019,30sleep20-21swapper/207:16:022
2174227913,54sleep021791-21latency07:11:040
3267227811,54sleep213441-21snmpd09:25:582
2895827820,38sleep329017-21sendmail-msp10:55:593
2484927816,40sleep324825-21latency_hist12:26:003
2261127815,26sleep00-21swapper/012:16:290
2751327715,43sleep027516-21cron09:10:580
2150727723,37sleep221574-21munin-node07:11:012
2555727614,41sleep225597-21ps12:26:282
219122768,50sleep20-21swapper/210:31:352
131427628,25sleep00-21swapper/007:46:280
131427628,25sleep00-21swapper/007:46:280
891327525,22sleep30-21swapper/309:51:313
2131927517,35sleep221293-21latency_hist08:51:022
2028427522,30sleep30-21swapper/307:06:183
98282748,17sleep00-21swapper/009:56:090
804027416,43sleep28085-21latency_hist09:51:022
71942747,54sleep00-21swapper/011:28:040
71942747,54sleep00-21swapper/011:28:040
2618427412,48sleep026194-21vmstat07:21:330
224892740,7sleep30-21swapper/307:11:223
3247127321,24sleep30-21swapper/311:06:203
2663127321,35sleep326653-21netstat09:06:253
1998527316,40sleep220009-21/usr/sbin/munin10:27:232
132352734,57sleep00-21swapper/008:25:530
2527027211,45sleep025358-21sed09:01:300
2527027211,45sleep025358-21sed09:01:300
2373027210,48sleep023732-21sshd07:16:120
21502729,48sleep22305-21cat07:51:022
21502729,48sleep22305-21cat07:51:022
936927114,24sleep20-21swapper/208:12:102
807427125,24sleep30-21swapper/311:31:223
176327121,31sleep21791-21munin-run09:30:592
299627023,23sleep30-21swapper/307:51:323
299627023,23sleep30-21swapper/307:51:323
2964627016,38sleep229735-21apt-config07:36:002
2049327012,44sleep220497-21diskstats12:11:182
152962709,50sleep00-21swapper/011:54:160
152962709,50sleep00-21swapper/011:54:160
882226924,28sleep08336-21apt-get08:11:110
469626916,25sleep30-21swapper/311:21:173
308112696,10sleep20-21swapper/209:20:012
282326917,33sleep32883-21latency_hist11:16:013
2586026915,35sleep025879-21awk10:45:590
198192695,11sleep00-21swapper/012:07:120
1868426926,29sleep20-21swapper/207:01:172
1664726924,29sleep30-21swapper/311:56:333
1589526915,33sleep316048-21latency_hist10:16:023
1548726921,23sleep00-21swapper/008:31:260
955426821,28sleep20-21swapper/211:36:222
955426821,28sleep20-21swapper/211:36:222
673326822,28sleep36724-21apt-get09:46:143
3200926822,21sleep30-21swapper/309:21:443
2692126812,25sleep20-21swapper/207:26:192
107662688,10sleep30-21swapper/309:56:323
703926712,41sleep07058-21apt-get08:06:100
437126719,33sleep34417-21cat11:21:013
437126719,33sleep34417-21cat11:21:013
2927226718,30sleep329267-21apt-get10:56:113
2767126717,31sleep20-21swapper/210:51:092
1235626723,27sleep312379-21/usr/sbin/munin08:21:243
1011626712,39sleep310118-21cron08:15:583
592026616,24sleep20-21swapper/211:26:012
2931126617,22sleep00-21swapper/009:15:590
2363526623,27sleep223789-21/usr/sbin/munin10:37:292
1894126614,39sleep318991-21fschecks_count12:06:183
1858626620,20sleep00-21swapper/007:01:110
1266126626,25sleep20-21swapper/208:21:312
2419926513,28sleep20-21swapper/207:16:312
1851726510,39sleep218528-21df_inode08:41:172
389626411,37sleep23965-21users11:16:322
389626411,37sleep23965-21users11:16:322
2549226417,29sleep325536-21awk12:26:253
2361826416,23sleep00-21swapper/012:21:170
1739026417,31sleep317402-21/usr/sbin/munin12:01:183
1739026417,31sleep317402-21/usr/sbin/munin12:01:183
1169026420,20sleep30-21swapper/310:01:243
1112426415,32sleep211133-21modprobe10:00:592
3932637,12sleep00-21swapper/007:44:060
2721326312,35sleep027262-21ntpq07:26:250
252726317,31sleep02593-21ps09:31:300
2242926318,20sleep00-21swapper/010:36:010
1885426318,26sleep218846-21apt-get12:06:132
1807826324,18sleep30-21swapper/308:41:023
1109926317,27sleep20-21swapper/211:41:222
1109926317,27sleep20-21swapper/211:41:222
760326214,30sleep27673-21sensors_temp08:06:292
364326221,21sleep20-21swapper/207:55:592
364326221,21sleep20-21swapper/207:55:592
332526211,21sleep00-21swapper/011:16:220
332526211,21sleep00-21swapper/011:16:220
3209226225,21sleep20-21swapper/211:05:592
2337326217,30sleep323436-21sendmail_mailst10:37:263
1483726219,20sleep00-21swapper/008:31:010
1091926216,22sleep30-21swapper/311:41:193
1091926216,22sleep30-21swapper/311:41:193
47826124,19sleep20-21swapper/211:06:312
2813326119,25sleep30-21swapper/307:31:013
2313426116,19sleep00-21swapper/008:56:130
2313426116,19sleep00-21swapper/008:56:130
2097026113,23sleep30-21swapper/310:31:123
2009726119,21sleep00-21swapper/010:27:260
1650426117,20sleep00-21swapper/010:16:270
1106426116,30sleep211194-21sort08:16:302
87826015,22sleep30-21swapper/309:26:243
779226023,20sleep20-21swapper/211:31:172
442126014,18sleep00-21swapper/011:21:020
2984526021,17sleep00-21swapper/007:36:020
2574226019,22sleep30-21swapper/307:21:283
186542606,12sleep00-21swapper/010:22:410
1619226018,23sleep20-21swapper/210:16:212
1238226017,27sleep212446-21/usr/sbin/munin11:46:152
750225919,16sleep20-21swapper/211:31:012
560625917,25sleep25642-21fschecks_count08:01:202
3016825919,22sleep00-21swapper/007:36:110
2655025914,27sleep226630-21memory09:06:242
241672597,11sleep30-21swapper/308:57:573
241672597,11sleep30-21swapper/308:57:573
1370525918,19sleep30-21swapper/311:51:003
1150625916,19sleep00-21swapper/010:01:210
530825814,19sleep00-21swapper/008:01:020
2508725813,22sleep30-21swapper/309:01:253
2508725813,22sleep30-21swapper/309:01:253
2479425820,18sleep20-21swapper/209:01:202
2479425820,18sleep20-21swapper/209:01:202
770325715,25sleep07699-21apt-get11:31:110
733525719,18sleep30-21swapper/308:06:223
61342578,37sleep30-21swapper/309:42:023
3057125715,28sleep030590-21systemd-run11:01:000
3020425716,20sleep30-21swapper/307:36:183
181625714,17sleep00-21swapper/009:31:010
1482425714,27sleep314835-21systemd-run08:31:003
133972575,12sleep00-21swapper/011:47:500
1296425715,25sleep012956-21apt-get10:06:140
463925614,18sleep20-21swapper/207:56:312
274825616,22sleep20-21swapper/211:16:002
2664825614,24sleep226638-21apt-get12:31:162
2164825613,20sleep20-21swapper/208:51:182
176252565,40sleep00-21swapper/008:37:110
1416525614,25sleep314162-21apt-get11:51:123
1416525614,25sleep314162-21apt-get11:51:123
1092425613,27sleep010987-21/usr/sbin/munin08:16:280
372325517,16sleep00-21swapper/007:56:010
372325517,16sleep00-21swapper/007:56:010
2204925513,18sleep00-21swapper/008:51:280
2069425511,25sleep320778-21memory12:11:233
1451225514,23sleep214508-21apt-get10:11:122
1346225511,19sleep30-21swapper/310:06:293
676825416,17sleep20-21swapper/211:26:292
2710925413,16sleep30-21swapper/312:31:303
1699925413,24sleep217034-21memory08:36:242
1586025417,18sleep30-21swapper/308:31:333
864925317,21sleep00-21swapper/009:51:240
644025311,20sleep00-21swapper/009:45:590
578525315,19sleep20-21swapper/209:41:332
2689625319,18sleep00-21swapper/010:46:310
205725312,24sleep22051-21apt-get09:31:152
1998025315,22sleep320001-21apt-get08:46:113
1765625314,17sleep20-21swapper/210:21:182
3271525214,23sleep032724-21/usr/sbin/munin09:26:000
2125525210,25sleep00-21swapper/012:11:320
1701525217,20sleep017024-21modprobe12:00:580
1549425212,23sleep215506-21munin-run11:56:002
1549425212,23sleep215506-21munin-run11:56:002
147625211,24sleep01469-21apt-get11:11:130
1058925218,17sleep00-21swapper/011:41:000
1058925218,17sleep00-21swapper/011:41:000
588825114,18sleep30-21swapper/308:01:253
2615325113,22sleep326146-21apt-get10:46:143
2331425111,22sleep023346-21ntp_states10:37:230
1767325111,24sleep017277-21/usr/sbin/munin12:01:230
1767325111,24sleep017277-21/usr/sbin/munin12:01:230
369925016,17sleep20-21swapper/209:36:162
3220125012,22sleep332207-21ntp_states07:41:263
2206925014,22sleep322076-21df_inode12:16:153
2166125015,16sleep30-21swapper/308:51:193
551724913,20sleep05510-21apt-get08:01:140
2748124913,18sleep20-21swapper/210:51:012
1940224913,22sleep318922-21apt-get10:26:073
79902479,15sleep00-21swapper/009:51:000
2321224511,16sleep20-21swapper/208:56:152
2321224511,16sleep20-21swapper/208:56:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional