You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-24 - 09:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Fri Jul 24, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
190322757125,7530sleep017306-21kworker/0:223:58:090
42682481012,4783sleep129822-21kworker/1:119:43:351
32192246247,4605sleep032144-21apt-get21:13:560
17683258711,562sleep217722-21ntpq22:08:362
11973249219,456sleep311975-21sshd21:51:333
1121224756,454sleep011248-21ntpq21:48:350
28580246023,421sleep028594-21sshd22:43:330
11395245152,96sleep00-21swapper/023:33:270
30928234714,285sleep031068-21tail00:33:420
26305233841,244sleep026330-21awk19:13:100
13222232435,211sleep013024-1kworker/0:0H20:13:250
16558231121,275sleep016635-21wc23:48:410
16558231121,275sleep016635-21wc23:48:410
27935227414,204sleep028079-21unixbench_multi20:58:420
31397226718,221sleep331461-21fschecks_time22:53:293
6089226076,20sleep00-21swapper/019:48:420
30453224535,197sleep030534-21grep21:08:310
31448223350,122sleep00-21swapper/019:28:120
7831222747,110sleep00-21swapper/021:38:300
25180222616,197sleep025262-21fschecks_time22:33:280
28809222550,129sleep028810-21cpu21:03:260
28809222550,129sleep028810-21cpu21:03:260
8302222019,180sleep28366-21sort23:23:282
8302222019,180sleep28366-21sort23:23:282
23803221737,119sleep012-21rcu_sched22:28:320
13032221634,49sleep00-21swapper/020:13:130
22551221115,116sleep00-21swapper/020:43:240
1565522068,187sleep0364-21snmpd23:47:400
28406219864,81sleep00-21swapper/000:28:080
28406219864,81sleep00-21swapper/000:28:080
32307218946,21sleep00-21swapper/019:28:400
20589218830,122sleep020655-21ls22:18:310
1230218816,155sleep11340-21users22:58:441
1230218816,155sleep11340-21users22:58:441
3549218723,145sleep23695-21unixbench_singl21:23:432
291972175105,54sleep029272-21xargs19:18:430
1522421737,155sleep015258-21ntp_states20:18:340
615621719,146sleep00-21swapper/021:33:210
1505121710,39sleep215055-21unixbench_multi21:58:422
26860216815,139sleep126878-21basename00:23:091
26860216815,139sleep126878-21basename00:23:091
30838216739,23sleep00-21swapper/022:50:480
13354216624,125sleep20-21swapper/221:54:312
18516216210,63sleep00-21swapper/020:28:390
5267216143,63sleep00-21swapper/023:13:290
26290215528,87sleep026408-21timerandwakeup20:53:410
17313215069,40sleep00-21swapper/022:08:260
1396821469,25sleep00-21swapper/021:58:120
12597214526,102sleep112113-21apt-get21:53:211
7353214110,27sleep07502-21tail23:18:410
7353214110,27sleep07502-21tail23:18:410
478321419,120sleep3364-21snmpd23:12:593
2007921407,11sleep00-21swapper/022:17:340
2007921407,11sleep00-21swapper/022:17:340
375621385,81sleep00-21swapper/019:43:130
24629213345,52sleep01-21systemd20:48:410
206621329,65sleep0364-21snmpd23:03:280
1097721325,50sleep00-21swapper/023:32:340
2597213039,25sleep00-21swapper/021:23:120
2597213039,25sleep00-21swapper/021:23:120
4229212421,89sleep34422-21apt-config21:28:123
1110212454,16sleep00-21swapper/022:58:430
1110212454,16sleep00-21swapper/022:58:430
718221230,38sleep17185-21fschecks_time19:53:311
23677212024,73sleep323193-21/usr/sbin/munin19:03:373
22437211944,14sleep00-21swapper/000:08:230
26753211629,74sleep026795-21fschecks_count22:38:290
24498211621,26sleep10-21swapper/100:13:401
1755221167,98sleep0364-21snmpd20:27:210
1755221167,98sleep0364-21snmpd20:27:210
15473211630,66sleep115501-21munin-plugin-st22:03:101
13257211645,20sleep00-21swapper/023:38:350
966921157,10sleep00-21swapper/020:00:580
966921157,10sleep00-21swapper/020:00:580
5174211437,61sleep00-21swapper/021:28:530
15766211413,19sleep10-21swapper/122:03:261
7978211318,75sleep18030-21latency_hist23:23:101
7978211318,75sleep18030-21latency_hist23:23:101
185721135,44sleep00-21swapper/019:35:510
1765021120,7sleep10-21swapper/122:08:341
24478211143,43sleep124514-21munin-run19:08:101
2206721115,70sleep00-21swapper/020:40:250
817421106,70sleep00-21swapper/019:57:060
7604211012,76sleep37685-21cat19:53:423
404821096,69sleep00-21swapper/021:27:360
32064210913,78sleep232110-21awk19:28:342
3248210823,64sleep13264-21ls23:08:091
9439210711,55sleep09505-21idleruntime21:43:310
9109210611,77sleep19179-21/usr/sbin/munin19:58:401
9109210611,77sleep19179-21/usr/sbin/munin19:58:401
30715210614,70sleep130223-21/usr/sbin/munin00:33:391
1631621069,46sleep016382-21awk22:03:410
341021045,64sleep00-21swapper/019:40:350
28689210430,21sleep028691-21apt-get00:28:210
28689210430,21sleep028691-21apt-get00:28:210
22454210417,74sleep222507-21processes22:23:372
2908221030,24sleep129086-21unixbench_singl22:43:421
11436210314,72sleep311498-21sed20:08:113
26659210212,22sleep00-21swapper/019:13:180
31113210117,69sleep231262-21latency_hist22:53:122
2943229920,55sleep329565-21unixbench_singl00:28:443
2943229920,55sleep329565-21unixbench_singl00:28:443
182329912,26sleep31824-21sshd19:35:293
217429816,19sleep30-21swapper/323:03:313
2145829745,39sleep00-21swapper/000:03:460
3177329628,38sleep10-21swapper/121:13:121
314642947,12sleep00-21swapper/022:53:310
1944829336,39sleep00-21swapper/020:33:230
174229313,65sleep31756-21wc23:03:103
174229313,65sleep31756-21wc23:03:103
224342928,34sleep022441-21ntp_states22:23:350
1348229210,63sleep20-21swapper/223:38:422
1022429225,48sleep20-21swapper/220:03:302
2857029114,63sleep228647-21latency_hist21:03:122
2398629138,13sleep00-21swapper/000:13:230
1517629119,15sleep30-21swapper/320:18:323
3139529024,47sleep231428-21munin-run19:28:102
3189728912,18sleep20-21swapper/222:53:412
2391928917,21sleep10-21swapper/120:48:131
1625128913,51sleep315741-21/usr/sbin/munin22:03:403
258428813,61sleep32595-21munin-run21:23:113
258428813,61sleep32595-21munin-run21:23:113
3014028619,34sleep30-21swapper/319:23:263
959228526,38sleep39661-21/usr/sbin/munin21:43:343
693828516,55sleep36993-21cat23:18:323
693828516,55sleep36993-21cat23:18:323
203628519,46sleep12073-21munin-run19:38:101
1144928512,49sleep210817-21/usr/sbin/munin21:48:412
667528415,55sleep16763-21grep21:33:391
2377228428,29sleep10-21swapper/100:13:121
2101728412,58sleep221100-21sed22:18:412
2528328324,28sleep20-21swapper/200:18:092
884228220,37sleep28882-21latency19:58:332
884228220,37sleep28882-21latency19:58:332
35382818,15sleep00-21swapper/023:08:240
2577327914,43sleep125878-21tune2fs20:53:301
2452927929,26sleep20-21swapper/219:08:112
2062527911,53sleep020667-21true00:03:100
3150227823,37sleep331540-21sh00:38:093
2657727822,27sleep10-21swapper/119:13:151
1921327815,48sleep119278-21latency_hist20:33:121
77827717,47sleep2779-21cpu19:33:262
2947227721,40sleep329480-21modprobe22:48:093
2281427720,29sleep20-21swapper/220:43:342
1774027719,36sleep317255-21apt-get23:53:203
833527621,40sleep28453-21latency_hist19:58:132
269122768,17sleep00-21swapper/000:23:120
269122768,17sleep00-21swapper/000:23:120
2266827622,37sleep122810-21unixbench_multi22:23:401
197427620,36sleep22118-21tail21:18:462
197427620,36sleep22118-21tail21:18:462
1448827618,36sleep114513-21munin-run20:18:101
2772127418,40sleep127781-21/usr/sbin/munin20:58:391
1959927425,20sleep10-21swapper/122:13:431
1959927425,20sleep10-21swapper/122:13:431
614327314,43sleep36151-21/usr/sbin/munin21:33:203
500327310,13sleep20-21swapper/221:28:472
2835227311,29sleep20-21swapper/219:18:132
2070227317,41sleep220852-21latency_hist00:03:122
2797927222,29sleep30-21swapper/322:43:123
1761327220,32sleep317648-21basename20:28:093
1761327220,32sleep317648-21basename20:28:093
146312720,10sleep30-21swapper/321:58:353
984927126,30sleep20-21swapper/223:28:282
84352716,12sleep00-21swapper/023:23:300
84352716,12sleep00-21swapper/023:23:300
500927119,29sleep30-21swapper/321:28:483
2517727114,42sleep225183-21/usr/sbin/munin19:09:042
144782716,49sleep00-21swapper/021:58:330
1192327115,42sleep312049-21cat23:33:383
1020427110,46sleep00-21swapper/020:03:280
83827023,22sleep20-21swapper/222:58:372
83827023,22sleep20-21swapper/222:58:372
2868927027,23sleep10-21swapper/119:18:321
2700727021,26sleep20-21swapper/220:58:112
1724227015,38sleep117273-21munin-run23:53:101
1724227015,38sleep117273-21munin-run23:53:101
600426920,30sleep35425-21/usr/sbin/munin19:48:413
535526912,25sleep20-21swapper/223:13:312
3243026926,24sleep30-21swapper/321:14:233
2526426919,28sleep10-21swapper/122:33:291
354026824,23sleep20-21swapper/223:08:262
3181626817,34sleep131896-21readlink22:53:401
3003226818,26sleep30-21swapper/322:48:333
2228926817,29sleep30-21swapper/320:43:113
2222226818,24sleep10-21swapper/100:08:111
1678526813,38sleep116809-21ntp_states20:23:351
1678526813,38sleep116809-21ntp_states20:23:351
1319426814,33sleep30-21swapper/320:13:203
1013126814,29sleep30-21swapper/323:28:333
2543726721,24sleep30-21swapper/320:53:123
2313826713,32sleep322435-21/usr/sbin/munin00:08:423
1266926722,21sleep10-21swapper/123:38:121
2772326612,18sleep30-21swapper/300:23:393
2772326612,18sleep30-21swapper/300:23:393
2480826620,25sleep20-21swapper/220:48:432
2335726616,33sleep123498-21modprobe22:28:121
118552665,13sleep20-21swapper/223:33:372
3178026511,25sleep10-21swapper/119:28:291
2943126519,30sleep329553-21timerwakeupswit21:03:413
2943126519,30sleep329553-21timerwakeupswit21:03:413
1450526516,26sleep20-21swapper/223:43:262
813926410,25sleep20-21swapper/221:38:362
2985026420,21sleep30-21swapper/319:23:103
1923126412,29sleep20-21swapper/222:13:362
1923126412,29sleep20-21swapper/222:13:362
138126418,23sleep10-21swapper/121:18:331
138126418,23sleep10-21swapper/121:18:331
580226320,26sleep30-21swapper/323:13:403
55532637,14sleep10-21swapper/121:28:591
3054626316,31sleep130593-21ntp_states19:23:371
2931426320,25sleep20-21swapper/200:28:432
2931426320,25sleep20-21swapper/200:28:432
250442635,17sleep30-21swapper/319:08:253
1935726323,19sleep30-21swapper/323:58:253
1879626319,21sleep20-21swapper/220:28:412
1806926312,31sleep10-21swapper/123:54:311
1470026313,27sleep30-21swapper/323:43:313
1177026315,35sleep111874-21grep20:08:291
925426217,19sleep30-21swapper/319:58:423
925426217,19sleep30-21swapper/319:58:423
2160326218,26sleep221736-21unixbench_multi00:03:482
1107526226,20sleep10-21swapper/123:33:121
80322619,26sleep28169-21cat23:23:112
3227626118,27sleep232282-21df_inode21:14:062
2840326118,19sleep30-21swapper/319:18:143
164462618,24sleep20-21swapper/223:48:392
164462618,24sleep20-21swapper/223:48:392
1298226115,27sleep30-21swapper/320:13:123
1070326120,26sleep10-21swapper/120:03:411
83212608,12sleep30-21swapper/321:38:413
3075426015,16sleep20-21swapper/200:33:402
2899026016,26sleep129054-21/usr/sbin/munin21:03:311
2899026016,26sleep129054-21/usr/sbin/munin21:03:311
2754526019,22sleep20-21swapper/200:23:342
2754526019,22sleep20-21swapper/200:23:342
261326019,22sleep30-21swapper/319:38:323
2134526015,27sleep121348-21/usr/sbin/munin22:18:441
199326013,32sleep22006-21cpu23:03:252
1605226022,17sleep30-21swapper/323:48:273
1605226022,17sleep30-21swapper/323:48:273
675625916,26sleep16759-21/usr/sbin/munin19:53:111
451325921,19sleep30-21swapper/319:43:413
309442597,15sleep30-21swapper/321:08:423
2642725917,27sleep326491-21latency_hist22:38:123
2409425918,28sleep324114-21grep00:13:303
184412598,15sleep30-21swapper/322:11:313
1480525912,29sleep214854-21/usr/sbin/munin20:18:232
1342725912,32sleep313573-21sed21:54:323
3076725816,22sleep20-21swapper/221:08:372
2319925814,28sleep323318-21/usr/sbin/munin20:43:433
2110125819,18sleep30-21swapper/322:18:423
1911825822,19sleep20-21swapper/223:58:112
1339525813,25sleep10-21swapper/123:38:411
52925713,19sleep30-21swapper/319:33:113
2924725716,26sleep10-21swapper/100:28:421
2924725716,26sleep10-21swapper/100:28:421
2703625716,18sleep30-21swapper/320:58:123
2633825718,23sleep126355-21munin-run19:13:111
2228525714,26sleep20-21swapper/220:43:102
1460125718,23sleep10-21swapper/123:43:291
118222578,37sleep10-21swapper/121:49:371
1000625716,25sleep10-21swapper/121:43:441
867125611,31sleep18677-21ntp_states23:23:351
867125611,31sleep18677-21ntp_states23:23:351
544725614,25sleep25444-21apt-get19:48:242
479725614,28sleep14841-21systemd-run23:13:091
2736125612,22sleep10-21swapper/122:38:421
2516725614,18sleep20-21swapper/222:33:262
2319225614,22sleep00-21swapper/019:03:210
1894025617,20sleep30-21swapper/322:13:293
1894025617,20sleep30-21swapper/322:13:293
188232565,11sleep30-21swapper/320:28:503
1288925616,21sleep30-21swapper/323:38:263
1015225615,24sleep210198-21users21:43:452
310192557,13sleep10-21swapper/122:53:071
2538725512,29sleep125417-21munin-run20:53:101
676425416,15sleep20-21swapper/221:33:402
2606425416,18sleep30-21swapper/300:18:393
2606425416,18sleep30-21swapper/300:18:393
2353625412,19sleep20-21swapper/222:28:262
2353225414,24sleep323528-21apt-get22:28:233
232925412,25sleep12348-21apt-get19:38:221
2129225418,22sleep30-21swapper/300:03:413
1650725414,21sleep20-21swapper/222:03:432
1321925413,24sleep213215-21apt-get20:13:232
563725316,17sleep10-21swapper/123:13:381
3251725315,21sleep10-21swapper/121:14:251
2841025310,23sleep128457-21munin-run00:28:101
2841025310,23sleep128457-21munin-run00:28:101
2572025313,25sleep225806-21perl00:18:302
2572025313,25sleep225806-21perl00:18:302
222762537,36sleep10-21swapper/120:43:051
1174325313,25sleep211749-21df_inode20:08:262
521025216,19sleep20-21swapper/219:48:122
2821525213,23sleep228211-21apt-get22:43:232
2752025212,22sleep227573-21users22:38:432
2253725214,25sleep222435-21/usr/sbin/munin00:08:292
2078825216,22sleep120925-21cat20:38:121
1480425213,23sleep114798-21apt-get20:18:221
2896325113,24sleep228994-21nfs_client19:18:372
77222508,15sleep10-21swapper/121:38:231
2205725012,23sleep322064-21df_inode22:23:263
1945025020,16sleep20-21swapper/220:33:252
344824915,19sleep10-21swapper/121:23:421
292292498,15sleep20-21swapper/221:03:362
292292498,15sleep20-21swapper/221:03:362
240424912,19sleep20-21swapper/219:38:272
1635524911,22sleep216349-21apt-get20:23:242
1635524911,22sleep216349-21apt-get20:23:242
1984524810,22sleep119891-21ps23:58:391
858424613,19sleep38628-21ntpq23:23:333
858424613,19sleep38628-21ntpq23:23:333
2770624412,18sleep30-21swapper/320:58:373
674024310,15sleep20-21swapper/223:18:272
194942428,18sleep30-21swapper/320:33:263
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional