You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-28 - 18:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Jun 28, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2420227457,722sleep024263-21perl08:20:030
2420227457,722sleep024263-21perl08:20:030
2883826108,588sleep219370-1kworker/2:2H12:00:082
26908252047,78sleep026960-21latency_hist08:29:440
26908252047,78sleep026960-21latency_hist08:29:440
625723086,266sleep00-21swapper/012:30:100
12858229718,265sleep212908-21unin-run07:44:442
24372229320,260sleep324409-21ps11:45:113
2176122518,230sleep021797-21ntpq09:55:080
766422495,212sleep00-21swapper/010:54:560
766422495,212sleep00-21swapper/010:54:560
1750022468,223sleep01-21systemd07:59:440
9537222316,191sleep29626-21threads09:15:132
9537222316,191sleep29626-21threads09:15:132
15259221946,63sleep00-21swapper/009:35:010
12996221956,49sleep00-21swapper/011:10:110
7380221514,92sleep07506-21latency_hist10:54:440
9544221414,146sleep09512-21latency_hist07:34:440
19646220844,57sleep00-21swapper/008:05:050
7164219144,95sleep07293-21latency_hist09:09:450
22177218911,160sleep19017-1kworker/1:2H08:14:431
7631218715,17sleep00-21swapper/012:35:070
315021779,58sleep00-21swapper/010:40:020
25363216515,134sleep125338-21latency_hist08:24:441
25363216515,134sleep125338-21latency_hist08:24:441
15074216410,53sleep015124-21awk07:50:070
8067216312,133sleep00-21swapper/009:10:140
25868216330,118sleep026021-21latency_hist10:09:460
14093215813,127sleep013597-21/usr/sbin/munin09:30:090
208021538,132sleep02162-21cut07:10:130
1531214744,90sleep01630-21cat08:50:110
3191214410,117sleep00-21swapper/007:14:500
1120221437,96sleep011415-21cat07:39:450
8320214019,96sleep28333-21fschecks_count07:30:022
15821214018,110sleep015825-21apt-config11:19:560
1394213618,96sleep10-21swapper/110:34:581
25459213413,102sleep00-21swapper/011:49:560
1118921347,93sleep011196-21sort11:06:030
919221278,62sleep00-21swapper/010:59:570
919221278,62sleep00-21swapper/010:59:570
3819212733,79sleep03824-21munin-run08:59:440
1393212735,50sleep00-21swapper/010:34:570
12713212413,97sleep112766-21cat11:10:041
1254721245,107sleep00-21swapper/007:40:420
27314212338,23sleep00-21swapper/008:30:040
1340212332,41sleep00-21swapper/007:09:450
11834212230,57sleep011841-21munin-run09:24:430
607621216,103sleep00-21swapper/007:20:350
2500021218,102sleep0364-21snmpd11:47:300
11016212114,20sleep30-21swapper/311:05:213
514821196,43sleep00-21swapper/012:26:060
1638211945,29sleep01742-21timerandwakeup12:15:120
1638211945,29sleep01742-21timerandwakeup12:15:120
4034211719,77sleep24210-21latency_hist08:59:462
26962211719,81sleep127102-21cat08:29:451
13379211733,72sleep0364-21snmpd09:29:440
4143211615,87sleep14163-21apt-get12:24:541
2882021158,22sleep00-21swapper/011:59:580
22158211521,75sleep322307-21latency_hist11:39:463
5103211417,84sleep05248-21head09:01:050
10984211427,60sleep110976-21apt-get11:04:591
618211312,62sleep0644-21munin-run08:49:420
30923211311,80sleep00-21swapper/010:25:040
1021521139,64sleep010282-21modprobe09:19:430
1021521139,64sleep010282-21modprobe09:19:430
30655211112,81sleep330694-21/usr/sbin/munin08:40:083
2283821108,19sleep00-21swapper/008:15:060
2283821108,19sleep00-21swapper/008:15:060
19557211021,48sleep00-21swapper/009:49:430
5691210820,51sleep00-21swapper/010:48:340
28504210811,43sleep00-21swapper/011:59:450
9587210612,80sleep29625-21ntpq11:00:072
9587210612,80sleep29625-21ntpq11:00:072
8266210613,76sleep28386-21timerwakeupswit10:55:102
8266210613,76sleep28386-21timerwakeupswit10:55:102
2235210417,65sleep20-21swapper/208:54:452
14594210417,47sleep00-21swapper/011:15:110
2364210314,70sleep30-21swapper/307:10:163
18035210237,12sleep00-21swapper/011:25:130
19066210017,68sleep319179-21apt-key11:29:483
17409210016,64sleep317525-21timerwakeupswit09:40:133
15125210013,67sleep10-21swapper/107:50:081
12288210016,22sleep212298-21munin-run11:09:442
36629911,73sleep3438-21sensors_temp10:30:123
36629911,73sleep3438-21sensors_temp10:30:123
539929815,67sleep25114-21/usr/sbin/munin07:20:052
1254329620,61sleep312564-21awk09:25:113
715829514,62sleep26667-21/usr/sbin/munin07:25:132
2758029511,71sleep227647-21switchtime11:55:112
2132529515,65sleep30-21swapper/308:10:083
624129424,54sleep36279-21ntpq09:05:083
285129321,47sleep12881-21munin-plugin-st07:14:421
2325329315,61sleep323266-21vmstat11:40:143
1794829335,11sleep00-21swapper/009:43:570
3085429111,58sleep130357-21/usr/sbin/munin12:05:121
1597329131,39sleep315368-21apt-get11:19:593
107829033,11sleep00-21swapper/010:34:250
107829033,11sleep00-21swapper/010:34:250
2946028915,57sleep329585-21cat12:00:293
257862890,8sleep10-21swapper/110:09:421
168028919,54sleep21695-21cut07:10:032
2078928819,52sleep220783-21apt-get11:34:562
1379228824,45sleep313816-21munin-run11:14:423
2367528717,53sleep123814-21latency_hist11:44:441
78328629,44sleep0830-21cat12:14:440
297328616,50sleep33054-21/usr/sbin/munin08:55:153
1927328622,12sleep00-21swapper/011:30:220
1051128626,36sleep310679-21gpgv11:04:443
1051128626,36sleep310679-21gpgv11:04:443
392928520,18sleep30-21swapper/312:24:443
2865928521,13sleep00-21swapper/010:16:480
131512859,17sleep00-21swapper/007:44:580
336328423,29sleep20-21swapper/207:14:522
1594828426,34sleep20-21swapper/211:19:582
102428412,17sleep10-21swapper/112:14:571
102428412,17sleep10-21swapper/112:14:571
669228320,47sleep00-21swapper/007:24:590
647428317,38sleep36613-21latency_hist07:24:463
315082835,46sleep01388-21cyclictest12:08:010
2279128310,16sleep00-21swapper/011:40:070
1830228323,26sleep30-21swapper/308:00:113
1830228323,26sleep30-21swapper/308:00:113
240142825,43sleep00-21swapper/010:01:540
2038628220,12sleep00-21swapper/009:50:110
2670928117,15sleep00-21swapper/010:10:140
2002528114,53sleep320061-21/usr/sbin/munin09:50:023
35192805,40sleep00-21swapper/012:20:130
3137328024,15sleep00-21swapper/008:42:400
2246028011,51sleep222561-21fschecks_time11:40:012
1651428016,45sleep216583-21latency07:55:052
303682795,40sleep00-21swapper/010:23:310
2879327916,51sleep228867-21fschecks_time08:35:072
1790327920,43sleep117944-21fschecks_count08:00:011
1790327920,43sleep117944-21fschecks_count08:00:011
598427817,40sleep15686-21/usr/sbin/munin12:30:031
509227720,45sleep35098-21cat09:01:043
3036227711,30sleep10-21swapper/108:40:021
2147827716,45sleep321549-21fschecks_time09:55:033
3129527623,32sleep10-21swapper/110:25:131
2878127617,13sleep00-21swapper/008:35:050
2531127619,22sleep30-21swapper/308:24:423
2531127619,22sleep30-21swapper/308:24:423
1577727612,51sleep315854-21xargs09:35:123
1370927615,41sleep313766-21grep09:29:593
2296927521,37sleep123045-21sendmail_mailtr08:15:111
2296927521,37sleep123045-21sendmail_mailtr08:15:111
1913327538,19sleep00-21swapper/008:04:440
1913327538,19sleep00-21swapper/008:04:440
158262759,15sleep10-21swapper/111:19:571
1568627518,41sleep215776-21mailstats09:35:112
74032749,48sleep10-21swapper/109:09:591
3191327417,39sleep131990-21tune2fs08:45:031
2621727415,45sleep226325-21cat08:25:132
2621727415,45sleep226325-21cat08:25:132
213752749,17sleep00-21swapper/008:10:090
945327312,41sleep19534-21mailstats09:15:121
945327312,41sleep19534-21mailstats09:15:121
761227318,39sleep17607-21apt-get10:54:531
761227318,39sleep17607-21apt-get10:54:531
342427313,44sleep20-21swapper/210:40:082
564927221,27sleep10-21swapper/107:20:121
1551027218,29sleep10-21swapper/111:19:451
833727116,40sleep38425-21fschecks_time07:30:033
518227118,37sleep25313-21sed10:45:132
2175527129,31sleep00-21swapper/011:35:550
1335627114,28sleep10-21swapper/107:45:031
2582827011,45sleep325863-21cat10:09:453
133092705,16sleep20-21swapper/209:29:222
126727023,26sleep21295-21munin-run07:09:432
387926920,34sleep13926-21systemd-run12:24:421
3250226929,20sleep10-21swapper/112:10:121
3227026918,26sleep10-21swapper/110:29:541
3227026918,26sleep10-21swapper/110:29:541
3150126916,30sleep330724-21/usr/sbin/munin10:25:153
25882699,13sleep20-21swapper/212:19:502
690126814,21sleep30-21swapper/307:25:063
2649926816,20sleep20-21swapper/210:10:082
2516826819,32sleep225201-21munin-run11:49:422
783526718,31sleep27904-21/usr/sbin/munin12:35:132
571226715,38sleep25778-21cpuspeed12:29:562
1927626716,31sleep219231-21apt-get11:30:242
184032679,11sleep20-21swapper/211:25:432
141562676,18sleep20-21swapper/207:46:292
1360226722,23sleep30-21swapper/307:45:093
955826621,25sleep10-21swapper/111:00:061
955826621,25sleep10-21swapper/111:00:061
799726619,33sleep11-21systemd07:29:451
606826621,19sleep10-21swapper/110:49:571
390726623,25sleep14032-21find08:59:451
3193026616,34sleep331996-21cpuspeed12:09:583
2792326611,40sleep327924-21sshd11:55:203
2367426620,26sleep20-21swapper/210:00:152
2062826630,21sleep10-21swapper/108:09:421
2029626614,37sleep120302-21ntp_states09:50:081
555326523,23sleep20-21swapper/209:04:432
3137026522,26sleep231500-21sort10:25:142
1453726512,40sleep214546-21ntpq11:15:092
1276726516,27sleep20-21swapper/211:10:052
797926414,26sleep30-21swapper/310:55:033
797926414,26sleep30-21swapper/310:55:033
790526417,20sleep30-21swapper/312:35:143
709926419,31sleep17128-21awk09:09:421
547726421,27sleep35534-21latency_hist12:29:453
3098526420,21sleep30-21swapper/312:05:143
3091426417,24sleep20-21swapper/212:05:132
2928226418,21sleep10-21swapper/110:20:001
2105426414,33sleep121070-21/usr/sbin/munin11:35:031
1959826418,32sleep119617-21latency_hist09:49:441
163126418,28sleep11773-21/usr/sbin/munin08:50:121
1533826416,28sleep215465-21/usr/sbin/munin07:50:132
1422926421,30sleep214309-21timerandwakeup09:30:112
1121526410,33sleep111366-21unixbench_multi09:20:121
35226311,37sleep2354-21sshd08:46:212
151626318,23sleep20-21swapper/212:15:102
151626318,23sleep20-21swapper/212:15:102
1161026311,39sleep30-21swapper/307:40:043
1157026317,27sleep111589-21apt-get07:39:541
1112026324,24sleep31-21systemd07:39:443
425626215,25sleep10-21swapper/108:59:531
2870926212,24sleep30-21swapper/308:35:043
132226220,21sleep30-21swapper/308:50:063
2939226117,19sleep30-21swapper/310:20:023
2506126116,19sleep20-21swapper/210:05:112
2254826114,31sleep222554-21diskstats08:15:002
2254826114,31sleep222554-21diskstats08:15:002
2057026121,24sleep220704-21sed09:50:132
1643126122,18sleep10-21swapper/109:39:421
1479326113,17sleep30-21swapper/307:50:023
109926115,25sleep30-21swapper/312:14:593
109926115,25sleep30-21swapper/312:14:593
661826020,19sleep30-21swapper/310:50:113
624926017,27sleep26307-21/usr/sbin/munin10:50:022
582526018,25sleep10-21swapper/110:49:441
55226018,26sleep3632-21timerandwakeup07:05:123
51002606,12sleep20-21swapper/212:25:322
302502607,15sleep20-21swapper/208:39:552
2888126016,24sleep30-21swapper/310:19:423
286272605,12sleep10-21swapper/111:59:461
2807226021,25sleep228102-21proc_pri10:15:092
2463226021,18sleep20-21swapper/208:20:132
2463226021,18sleep20-21swapper/208:20:132
1972326010,29sleep10-21swapper/111:31:031
577925917,24sleep30-21swapper/310:49:423
35582599,13sleep10-21swapper/107:14:591
313125920,18sleep30-21swapper/307:14:493
282362596,42sleep20-21swapper/208:32:282
2417225913,25sleep10-21swapper/111:45:051
1732325921,18sleep20-21swapper/209:40:122
305925820,18sleep30-21swapper/310:40:013
1485725819,22sleep10-21swapper/109:34:211
1412925818,19sleep30-21swapper/311:14:583
1052525813,25sleep110667-21grep07:35:141
630725723,20sleep20-21swapper/209:05:112
2995525715,28sleep129983-21munin-run08:39:421
2992625718,20sleep20-21swapper/210:20:142
1624625721,18sleep30-21swapper/307:54:583
839925624,17sleep20-21swapper/209:11:212
1962025617,17sleep20-21swapper/209:49:452
894725515,24sleep38942-21apt-get09:14:573
894725515,24sleep38942-21apt-get09:14:573
3238725516,18sleep10-21swapper/107:04:541
2591525519,16sleep30-21swapper/311:50:073
2423225517,23sleep124257-21munin-run10:04:421
1827025515,23sleep30-21swapper/309:44:533
115592554,13sleep20-21swapper/209:21:432
774825416,15sleep10-21swapper/112:35:121
427325415,22sleep14283-21munin-run10:44:441
3258525413,25sleep232711-21sed12:10:132
2826525415,25sleep128323-21munin-run11:59:421
276625416,16sleep30-21swapper/310:39:453
2685925415,16sleep30-21swapper/308:29:433
2685925415,16sleep30-21swapper/308:29:433
2596125420,18sleep10-21swapper/111:50:091
2553425415,23sleep225540-21diskstats11:49:582
2219925417,22sleep222226-21munin-run08:14:442
2199825416,23sleep20-21swapper/209:55:142
1780725416,22sleep217810-21apt-get07:59:542
1780725416,22sleep217810-21apt-get07:59:542
1624425413,25sleep116237-21apt-get07:54:561
2788325318,17sleep10-21swapper/110:15:041
2303325313,24sleep323042-21df_inode10:00:003
2255525314,21sleep30-21swapper/308:15:013
2255525314,21sleep30-21swapper/308:15:013
1790725313,23sleep117913-21ntp_states11:25:101
1713925314,20sleep10-21swapper/109:40:071
1340325316,16sleep10-21swapper/109:29:451
1026525313,18sleep20-21swapper/207:35:112
3181925214,23sleep331815-21apt-get08:44:573
783525113,22sleep37857-21ntp_states09:10:093
2724125114,20sleep30-21swapper/308:30:033
2683825117,17sleep10-21swapper/110:10:151
2139125113,21sleep121386-21apt-get09:54:571
204012514,11sleep30-21swapper/308:06:453
23202504,14sleep20-21swapper/210:35:242
1875925013,23sleep318766-21gpgv11:29:443
2560024914,19sleep10-21swapper/108:24:561
204882496,32sleep20-21swapper/208:07:532
848724812,21sleep18506-21irqstats07:30:051
577924712,20sleep35785-21df12:29:573
110124715,19sleep21148-21systemd-run10:34:432
110124715,19sleep21148-21systemd-run10:34:432
2615024611,18sleep326183-21cut10:10:013
1060024612,19sleep310606-21df_inode09:19:583
275722419,16sleep10-21swapper/108:30:091
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional