You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-09 - 22:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat May 09, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
317728118,790sleep33213-21ntpq12:05:433
29079274815,714sleep229110-21sshd11:45:072
15770229412,98sleep315909-21latency_hist11:04:453
9320227493,160sleep30-21swapper/307:15:143
235422637,198sleep32400-21ntpq10:20:103
235422637,198sleep32400-21ntpq10:20:103
21515224838,22sleep30-21swapper/307:55:083
21515224838,22sleep30-21swapper/307:55:083
19994224317,205sleep020136-21unixbench_multi11:15:160
6483223123,55sleep36551-21latency_hist07:09:453
19887222515,184sleep30-21swapper/307:50:083
19887222515,184sleep30-21swapper/307:50:083
259152216102,24sleep30-21swapper/308:10:023
259152216102,24sleep30-21swapper/308:10:023
13858220657,111sleep30-21swapper/309:14:563
935220311,131sleep31006-21switchtime10:15:143
32245219945,62sleep30-21swapper/308:30:053
20441219912,148sleep320503-21fschecks_time09:35:033
8056219814,138sleep38196-21latency_hist07:14:463
8056219814,138sleep38196-21latency_hist07:14:463
27249219838,53sleep30-21swapper/311:40:033
24794219719,132sleep324862-21latency_hist09:49:453
5548218715,71sleep3284-21dbus-daemon08:49:443
15234218511,157sleep115174-21latency_hist09:19:471
2942621768,127sleep329463-21modprobe10:04:443
2287021678,121sleep322917-21ntpq11:25:083
3187421669,43sleep331241-21/usr/sbin/munin10:10:173
56012165102,45sleep30-21swapper/310:30:163
12545216416,124sleep312578-21grep10:54:433
17927215711,133sleep317985-21idleruntime11:10:023
22856215510,130sleep222869-21netstat11:25:072
8200215012,21sleep08364-21users07:14:470
16816215013,78sleep316932-21apt-config09:24:453
16816215013,78sleep316932-21apt-config09:24:453
18676214530,97sleep30-21swapper/307:45:153
22252214120,94sleep00-21swapper/011:24:450
2415214035,69sleep32562-21sed12:04:463
2415214035,69sleep32562-21sed12:04:463
16446213850,19sleep30-21swapper/311:05:263
16446213850,19sleep30-21swapper/311:05:263
1265621350,36sleep012660-21unixbench_multi07:25:160
5932213419,101sleep16032-21fschecks_time12:15:021
11537213350,38sleep311549-21sh07:24:433
11537213350,38sleep311549-21sh07:24:433
15838213229,69sleep314900-21/usr/sbin/munin07:35:173
26844213036,76sleep326862-21cron11:39:423
26844213036,76sleep326862-21cron11:39:423
11790213057,30sleep311802-21sh12:34:433
5567212928,64sleep30-21swapper/312:14:443
8739212829,59sleep30-21swapper/312:24:453
1086321278,61sleep3363-21snmpd10:48:053
2535021269,98sleep325464-21latency_hist11:34:463
2535021269,98sleep325464-21latency_hist11:34:463
17758212610,80sleep317769-21ntpq09:25:513
12580212518,27sleep20-21swapper/212:35:142
27354212310,98sleep38119-21munin-node08:14:553
30949212113,44sleep331091-21grep11:50:163
30275212123,33sleep10-21swapper/108:24:461
2511321206,13sleep30-21swapper/309:50:023
19310212022,85sleep019352-21systemd-run07:49:440
17844212021,75sleep217923-21fschecks_time11:10:012
1159211622,77sleep21211-21sort08:35:012
20527211526,52sleep30-21swapper/311:19:073
8861211411,91sleep38976-21cat10:40:143
2757221145,49sleep30-21swapper/309:56:163
1132211417,80sleep18119-21munin-node08:34:561
18951211214,81sleep118964-21/usr/sbin/munin07:45:171
14215211220,73sleep114356-21latency_hist10:59:461
14215211220,73sleep114356-21latency_hist10:59:461
761921115,11sleep30-21swapper/310:36:343
11830211010,81sleep30-21swapper/307:25:003
7999210816,78sleep18039-21ntpq08:55:071
26679210817,75sleep126783-21fschecks_time09:55:041
256421080,6sleep00-21swapper/012:04:460
256421080,6sleep00-21swapper/012:04:460
25406210812,74sleep225451-21/usr/sbin/munin09:50:092
22702210829,33sleep10-21swapper/107:59:551
8693210713,77sleep18732-21modprobe12:24:431
21167210716,23sleep121260-21memory11:20:061
14173210715,78sleep014214-21cut10:59:450
14173210715,78sleep014214-21cut10:59:450
8169210610,54sleep30-21swapper/308:55:133
18293210314,73sleep018362-21sensors_temp11:10:110
1730821020,7sleep20-21swapper/209:25:002
2190629929,11sleep30-21swapper/309:39:563
573929819,57sleep15946-21rm08:49:461
2511929715,62sleep025129-21/usr/sbin/munin09:50:030
123082967,48sleep30-21swapper/309:09:553
1059629626,15sleep30-21swapper/309:04:463
469829524,20sleep30-21swapper/310:28:593
469829524,20sleep30-21swapper/310:28:593
2981429521,12sleep30-21swapper/308:20:163
737929319,14sleep30-21swapper/308:54:453
323229314,62sleep00-21swapper/008:40:120
323229314,62sleep00-21swapper/008:40:120
641429233,36sleep21-21systemd07:09:442
252829223,16sleep30-21swapper/308:39:463
943729118,14sleep30-21swapper/312:25:103
92892916,46sleep30-21swapper/309:00:023
3002629113,61sleep030148-21latency_hist11:49:450
2580629116,32sleep00-21swapper/008:09:550
2580629116,32sleep00-21swapper/008:09:550
177532916,54sleep30-21swapper/307:44:383
126582918,17sleep30-21swapper/312:35:153
3152328924,43sleep131537-21modprobe11:54:431
240442898,21sleep324155-21date08:04:453
2080428927,32sleep20-21swapper/209:35:102
734228824,48sleep27378-21basename08:54:442
2473828812,57sleep224850-21/usr/sbin/munin11:30:162
2473828812,57sleep224850-21/usr/sbin/munin11:30:162
1571828824,19sleep20-21swapper/211:04:442
715928710,62sleep07222-21/usr/sbin/munin10:35:140
706928714,59sleep27091-21pluginstate10:35:122
70542875,74sleep00-21swapper/012:18:570
2187328714,61sleep221883-21apt.systemd.dai11:20:512
131642870,9sleep213163-21latency10:55:062
131642870,9sleep213163-21latency10:55:062
242862868,15sleep30-21swapper/308:05:023
192632868,17sleep30-21swapper/309:30:113
3182328515,19sleep30-21swapper/308:29:453
322442848,14sleep30-21swapper/311:55:093
317932840,7sleep10-21swapper/111:54:541
944228315,52sleep09443-21cron10:44:430
845128327,27sleep00-21swapper/007:14:510
1643828310,18sleep00-21swapper/009:20:590
1643828310,18sleep00-21swapper/009:20:590
604328223,43sleep15559-21apt-get08:49:541
561128224,25sleep00-21swapper/008:49:450
72728111,52sleep1323-21/usr/sbin/munin10:15:081
701228119,30sleep00-21swapper/007:10:060
2311828112,50sleep123234-21/usr/sbin/munin11:25:151
1086928022,29sleep10-21swapper/109:05:051
554727923,35sleep25564-21awk12:14:432
2815727921,31sleep10-21swapper/109:59:571
1350327921,32sleep10-21swapper/112:39:461
1096127920,38sleep211109-21grep07:20:142
1096127920,38sleep211109-21grep07:20:142
1067627927,24sleep20-21swapper/210:45:422
1308627834,26sleep20-21swapper/207:29:432
83832770,7sleep20-21swapper/207:14:482
232442778,51sleep023366-21latency_hist09:44:460
146762778,31sleep30-21swapper/307:34:453
118332775,15sleep00-21swapper/009:07:100
1126727722,22sleep20-21swapper/209:05:142
681927619,24sleep00-21swapper/008:50:420
2678427623,34sleep226823-21/usr/sbin/munin09:55:052
1583827621,26sleep20-21swapper/209:20:082
1583827621,26sleep20-21swapper/209:20:082
1515127626,34sleep215180-21munin-run09:19:442
554227518,31sleep20-21swapper/208:49:432
3602755,12sleep00-21swapper/011:56:270
1931227514,47sleep119382-21df11:15:011
136082756,27sleep30-21swapper/310:55:163
136082756,27sleep30-21swapper/310:55:163
200322736,15sleep00-21swapper/009:34:160
3022227220,30sleep230254-21munin-run08:24:432
674727124,24sleep20-21swapper/207:10:002
674727124,24sleep20-21swapper/207:10:002
524027123,21sleep10-21swapper/107:05:031
3215427117,12sleep20-21swapper/208:30:042
3195227122,24sleep00-21swapper/010:10:180
2919927110,45sleep30-21swapper/311:45:123
1936227113,42sleep219418-21cut07:49:462
1052927119,28sleep20-21swapper/212:29:572
37932705,10sleep30-21swapper/308:42:093
37932705,10sleep30-21swapper/308:42:093
1751927020,25sleep20-21swapper/211:09:452
1751927020,25sleep20-21swapper/211:09:452
221232695,11sleep30-21swapper/311:23:433
1611426921,34sleep116228-21sed09:20:151
1611426921,34sleep116228-21sed09:20:151
1568326919,35sleep115823-21uptime07:35:151
134926913,20sleep20-21swapper/212:00:092
134926913,20sleep20-21swapper/212:00:092
541626821,25sleep00-21swapper/007:05:060
3061126818,28sleep00-21swapper/008:25:060
305526810,41sleep13148-21memory12:05:411
2464726816,31sleep124736-21timerandwakeup11:30:151
2464726816,31sleep124736-21timerandwakeup11:30:151
239022687,48sleep00-21swapper/008:03:330
1722526820,20sleep00-21swapper/009:24:550
718426742,14sleep30-21swapper/312:19:453
578926721,29sleep15922-21head10:30:181
475626715,36sleep24757-21cron10:29:422
475626715,36sleep24757-21cron10:29:422
3020426732,21sleep030266-21mailstats10:05:130
1174226715,36sleep211752-21ntpq10:50:102
967626614,33sleep29187-21/usr/sbin/munin09:00:132
929726612,37sleep09309-21/usr/sbin/munin09:00:030
67032666,11sleep10-21swapper/107:09:471
67032666,11sleep10-21swapper/107:09:471
3017926610,37sleep30-21swapper/310:05:123
2786826622,19sleep10-21swapper/108:15:131
2774726619,24sleep00-21swapper/008:15:080
2474326616,32sleep124760-21awk09:49:431
2163826613,29sleep00-21swapper/007:55:130
2163826613,29sleep00-21swapper/007:55:130
1523126617,37sleep01-21systemd09:19:460
845226525,25sleep20-21swapper/210:40:042
262492656,14sleep20-21swapper/208:10:092
262492656,14sleep20-21swapper/208:10:092
2554226518,32sleep1284-21dbus-daemon08:09:441
1556726520,27sleep00-21swapper/007:35:140
1280026516,36sleep012832-21proc_pri09:10:130
1099526515,35sleep110996-21ntp_states12:30:131
106802657,13sleep00-21swapper/007:20:080
106802657,13sleep00-21swapper/007:20:080
854526415,32sleep18663-21sshd07:14:521
3263726412,35sleep032706-21/usr/sbin/munin08:30:140
3097226419,27sleep231003-21munin-run10:09:432
1797726414,27sleep10-21swapper/109:26:011
846126311,40sleep0363-21snmpd12:21:430
513926313,21sleep00-21swapper/010:30:050
44126320,20sleep00-21swapper/010:15:020
2311826319,29sleep223145-21/usr/sbin/munin08:00:082
1687426312,37sleep116897-21netstat07:40:081
1650026320,26sleep216497-21apt-get07:39:572
1346526316,29sleep213470-21/usr/sbin/munin07:30:032
999826220,19sleep10-21swapper/110:45:041
3181026218,28sleep231807-21apt-get11:54:562
311222627,38sleep10-21swapper/110:09:461
2398726216,28sleep224063-21sendmail_mailtr09:45:162
206822626,11sleep00-21swapper/007:52:270
206822626,11sleep00-21swapper/007:52:270
2011426216,32sleep120154-21cut09:34:451
1468826220,25sleep00-21swapper/007:34:460
922226113,33sleep29252-21irqstats12:25:052
1677726117,22sleep00-21swapper/007:40:070
1591126119,18sleep00-21swapper/011:04:460
1591126119,18sleep00-21swapper/011:04:460
1492326118,26sleep214918-21apt-get07:34:562
1439426112,27sleep10-21swapper/110:59:541
3071426015,27sleep10-21swapper/108:25:071
2947926015,18sleep20-21swapper/210:04:472
284072606,44sleep10-21swapper/111:44:291
2716726025,18sleep00-21swapper/011:40:000
1907526019,19sleep20-21swapper/209:30:062
898325916,25sleep00-21swapper/012:24:590
70892596,11sleep20-21swapper/212:19:232
3203625918,25sleep132055-21apt-get08:29:551
2950025914,25sleep00-21swapper/008:20:130
2762225914,19sleep20-21swapper/208:15:062
2430825917,28sleep024377-21sort11:30:060
2430825917,28sleep024377-21sort11:30:060
1986625917,24sleep20-21swapper/207:50:062
1986625917,24sleep20-21swapper/207:50:062
140725921,24sleep01429-21irqstats08:35:070
1314125914,20sleep10-21swapper/107:29:461
1053225916,28sleep010525-21apt-get12:29:590
958125817,17sleep10-21swapper/109:00:081
2086225821,17sleep10-21swapper/109:35:141
1907025820,19sleep10-21swapper/111:14:441
1173225816,30sleep111738-21ntp_states10:50:091
338225717,21sleep00-21swapper/012:06:000
2437025715,23sleep20-21swapper/208:05:052
144372576,10sleep10-21swapper/107:32:081
59242568,26sleep20-21swapper/210:30:192
358525621,17sleep10-21swapper/110:25:051
358525621,17sleep10-21swapper/110:25:051
349725615,25sleep03491-21apt-get10:25:000
349725615,25sleep03491-21apt-get10:25:000
2876925617,21sleep00-21swapper/010:00:120
2585525621,18sleep20-21swapper/211:35:052
2585525621,18sleep20-21swapper/211:35:052
229412565,39sleep20-21swapper/209:41:292
2193125611,28sleep10-21swapper/109:39:581
2179425616,25sleep221920-21sed07:55:152
2179425616,25sleep221920-21sed07:55:152
825425514,23sleep18263-21df_inode10:39:591
56425515,23sleep2621-21awk10:15:042
471025511,27sleep04742-21ntp_states12:10:080
428325513,25sleep14277-21apt-get12:09:571
3151125513,21sleep20-21swapper/210:10:082
2998525517,17sleep10-21swapper/110:05:061
2935225517,21sleep10-21swapper/111:45:141
2435525514,17sleep00-21swapper/008:05:030
2380225517,17sleep00-21swapper/011:29:460
2115425514,26sleep021257-21users09:35:170
130925518,20sleep10-21swapper/112:00:081
130925518,20sleep10-21swapper/112:00:081
1291125517,19sleep20-21swapper/209:10:152
1233225513,25sleep112327-21apt-get09:09:571
1179325515,21sleep00-21swapper/010:50:120
824125415,24sleep08332-21cat08:55:140
330625420,17sleep10-21swapper/108:40:131
330625420,17sleep10-21swapper/108:40:131
281225410,19sleep20-21swapper/212:05:012
265962549,33sleep10-21swapper/111:36:291
265962549,33sleep10-21swapper/111:36:291
2345425414,25sleep023471-21apt09:44:540
1208225413,24sleep012073-21apt-get12:35:000
897925313,24sleep18976-21apt-get12:24:551
3026325314,21sleep030258-21apt-get11:49:570
29562534,18sleep20-21swapper/210:21:312
29562534,18sleep20-21swapper/210:21:312
2719925313,26sleep027310-21timerandwakeup09:55:160
202302536,35sleep20-21swapper/211:15:222
873325216,20sleep28738-21systemd-run12:24:442
194925215,23sleep11977-21grep10:20:001
194925215,23sleep11977-21grep10:20:001
1496025212,24sleep015025-21swap11:00:140
1099325212,14sleep00-21swapper/010:49:440
2572525115,17sleep10-21swapper/109:50:161
2091025113,16sleep10-21swapper/107:54:461
2091025113,16sleep10-21swapper/107:54:461
2586125010,15sleep10-21swapper/108:10:001
2586125010,15sleep10-21swapper/108:10:001
194825013,19sleep00-21swapper/010:19:590
194825013,19sleep00-21swapper/010:19:590
1072825014,21sleep110734-21ntp_states07:20:091
1072825014,21sleep110734-21ntp_states07:20:091
339124916,20sleep23486-21timerwakeupswit08:40:142
339124916,20sleep23486-21timerwakeupswit08:40:142
3093124912,20sleep230969-21ntp_states08:25:122
2686924916,17sleep00-21swapper/011:39:430
2686924916,17sleep00-21swapper/011:39:430
1623424911,21sleep116197-21apt-get11:05:151
1623424911,21sleep116197-21apt-get11:05:151
786524811,20sleep10-21swapper/110:39:431
78492488,17sleep20-21swapper/208:55:042
2139524814,17sleep10-21swapper/107:55:061
2139524814,17sleep10-21swapper/107:55:061
3023924615,16sleep20-21swapper/211:49:552
275832408,15sleep20-21swapper/211:40:102
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional