You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-10 - 21:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Mon Aug 10, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
292929963,9sleep00-21swapper/007:09:110
284329774,18sleep20-21swapper/207:08:042
269529466,9sleep30-21swapper/307:06:093
272528566,15sleep10-21swapper/107:06:311
106132750,0sleep00-21swapper/009:18:360
198052540,0sleep00-21swapper/011:00:190
295292280,0sleep30-21swapper/310:46:353
308699260,26cyclictest0-21swapper/212:31:042
3080992523,1cyclictest0-21swapper/011:22:540
308699240,24cyclictest0-21swapper/210:34:202
308999230,22cyclictest21520-21kworker/3:010:06:223
308699230,23cyclictest0-21swapper/210:11:472
3084992219,2cyclictest0-21swapper/111:48:181
308999210,20cyclictest0-21swapper/310:40:523
308499200,3cyclictest0-21swapper/110:04:371
308999190,1cyclictest0-21swapper/309:16:053
308699190,19cyclictest0-21swapper/212:16:262
308699190,17cyclictest0-21swapper/208:45:132
3084991817,1cyclictest0-21swapper/110:13:441
3080991816,2cyclictest0-21swapper/012:27:200
308099180,1cyclictest0-21swapper/011:19:090
308999171,16cyclictest0-21swapper/311:14:593
308699170,17cyclictest0-21swapper/211:18:202
308699170,17cyclictest0-21swapper/210:25:022
308699170,16cyclictest0-21swapper/209:21:582
308499172,14cyclictest28085-21diskmemload12:22:491
308499172,14cyclictest28085-21diskmemload12:22:491
3084991717,0cyclictest0-21swapper/107:10:121
3084991716,1cyclictest0-21swapper/109:13:591
3080991717,0cyclictest0-21swapper/012:00:150
270592170,1sleep30-21swapper/310:25:433
244052170,0sleep30-21swapper/309:26:213
3089991615,1cyclictest21520-21kworker/3:011:21:323
3089991615,1cyclictest0-21swapper/310:18:073
3089991611,3cyclictest28080-21basename09:10:003
308699160,1cyclictest0-21swapper/209:34:032
308699160,16cyclictest0-21swapper/212:20:582
308699160,16cyclictest0-21swapper/212:20:582
308499160,16cyclictest0-21swapper/110:53:041
308099160,16cyclictest0-21swapper/012:16:230
308999152,13cyclictest0-21swapper/311:41:013
308699154,8cyclictest27577-21cups-browsed11:25:212
308699151,1cyclictest0-21swapper/210:21:202
308699150,15cyclictest0-21swapper/211:41:302
308499159,5cyclictest26089-21ls10:25:221
308499157,5cyclictest19491-21perl11:00:161
3084991512,2cyclictest29419-21sed08:05:001
308499151,12cyclictest28085-21diskmemload12:09:061
308499150,15cyclictest0-21swapper/112:30:251
308499150,11cyclictest27599-21kworker/u8:010:41:471
308099157,8cyclictest13-21ksoftirqd/009:20:130
308099153,5cyclictest225-21systemd-journal09:27:350
308999144,9cyclictest13270-21/usr/sbin/munin11:35:163
3089991413,1cyclictest27084-21kworker/3:212:26:523
3089991413,1cyclictest21520-21kworker/3:011:31:523
3089991413,1cyclictest21520-21kworker/3:011:27:323
3089991410,3cyclictest21459-21wc07:45:243
3089991410,2cyclictest8317-21tr07:20:143
308999140,14cyclictest0-21swapper/312:19:123
308999140,0cyclictest0-21swapper/311:47:463
3086991411,2cyclictest3993-21cron08:20:002
308699141,12cyclictest0-21swapper/210:46:592
308699140,1cyclictest0-21swapper/212:14:372
308699140,0cyclictest0-21swapper/209:29:172
308499141,9cyclictest141rcu_preempt11:35:171
3084991411,2cyclictest14444-21grep07:30:261
3084991410,3cyclictest31480-21grep09:50:171
308499140,13cyclictest0-21swapper/107:40:151
308499140,12cyclictest0-21swapper/111:08:461
3080991411,2cyclictest7307-21wc11:50:270
3080991410,2cyclictest17934-21tr10:40:150
308099140,2cyclictest0-21swapper/012:21:500
308099140,2cyclictest0-21swapper/012:21:500
5942130,0sleep10-21swapper/111:27:331
308999139,3cyclictest4837-21grep08:20:153
308999139,3cyclictest22956-21grep09:45:173
308999137,2cyclictest537-21snmpd07:19:103
3089991312,1cyclictest21520-21kworker/3:011:09:223
308999131,12cyclictest0-21swapper/310:23:533
3089991310,2cyclictest14726-21tail11:55:183
3089991310,2cyclictest11638-21grep08:35:153
308999130,0cyclictest0-21swapper/312:21:153
308999130,0cyclictest0-21swapper/312:21:153
308699139,3cyclictest13982-21cut09:40:132
308699133,6cyclictest0-21swapper/210:18:312
3086991310,2cyclictest7073-21cron12:30:002
3086991310,2cyclictest30451-21grep09:10:262
3086991310,2cyclictest27934-21head08:00:182
3086991310,2cyclictest17355-21tr12:35:212
3086991310,2cyclictest13529-21munin-plugin-st08:40:002
308499138,3cyclictest27104-21ssh10:45:191
308499132,10cyclictest28085-21diskmemload11:32:091
308499131,1cyclictest10460-21ssh11:52:431
3084991310,2cyclictest4496-21tr11:10:201
3084991310,2cyclictest25229-21grep09:00:271
3084991310,2cyclictest1166-21grep12:25:271
308499130,1cyclictest0-21swapper/110:35:021
308499130,1cyclictest0-21swapper/110:35:021
308499130,13cyclictest28085-21diskmemload10:59:511
308499130,12cyclictest0-21swapper/109:44:341
308099139,3cyclictest4603-21fgrep11:10:210
308099139,3cyclictest30652-21cat08:05:210
308099139,3cyclictest24109-21wc09:45:270
308099139,2cyclictest9081-21wc07:20:190
308099139,2cyclictest7264-21wc07:15:250
308099139,2cyclictest3894-21cat11:30:000
308099131,7cyclictest5057-21munin-run09:35:000
3080991312,1cyclictest13-21ksoftirqd/012:13:250
3080991311,2cyclictest13-21ksoftirqd/009:50:050
3080991310,2cyclictest4359-21cut08:20:150
3080991310,2cyclictest26589-21grep10:25:270
3080991310,2cyclictest21585-21grep07:45:240
3080991310,2cyclictest1919-21grep08:15:130
3080991310,2cyclictest13829-21wc07:30:180
3080991310,2cyclictest13512-21cron08:40:000
3080991310,2cyclictest12140-21tail07:25:280
3080991310,2cyclictest11184-21tail10:35:260
3080991310,2cyclictest11184-21tail10:35:260
308999129,2cyclictest7553-21grep09:55:183
308999129,2cyclictest6632-21tr12:10:143
308999129,2cyclictest3566-21unixbench_singl08:15:263
308999129,2cyclictest3494-21ssh10:50:233
308999129,2cyclictest30834-21wc09:30:203
308999129,2cyclictest30343-21head08:05:183
308999129,2cyclictest29392-21seq12:04:263
308999129,2cyclictest15467-21ls12:35:003
308999129,2cyclictest14937-21cron07:35:003
308999129,2cyclictest13362-21cron11:55:003
308999129,2cyclictest12108-21wc10:55:253
308999128,3cyclictest6915-21tr08:25:143
308999128,3cyclictest32606-21grep12:05:273
308999122,1cyclictest0-21swapper/309:39:013
3089991211,1cyclictest0-21swapper/310:01:453
3089991210,1cyclictest31614-21/usr/sbin/munin09:50:183
308699129,2cyclictest6508-21wc08:25:002
308699129,2cyclictest5036-21cat09:15:162
308699129,2cyclictest3812-21grep11:10:152
308699129,2cyclictest30963-21cut09:50:152
308699129,2cyclictest2871-21head10:50:182
308699129,2cyclictest23142-21dpkg09:00:002
308699129,2cyclictest23103-21grep12:00:192
308699129,2cyclictest22455-21grep07:50:132
308699129,2cyclictest19147-21gzip10:40:222
308699129,2cyclictest15329-21basename07:35:152
308699129,2cyclictest15023-21grep08:40:222
308699129,2cyclictest10114-21tr10:35:172
308699129,2cyclictest10114-21tr10:35:172
308699121,6cyclictest19606-21sh10:02:272
3086991212,0cyclictest0-21swapper/210:57:042
308699120,11cyclictest29595-21ssh11:05:452
308499129,2cyclictest8845-21head07:20:171
308499129,2cyclictest7937-21apt-get10:15:121
308499129,2cyclictest6355-21grep09:35:161
308499129,2cyclictest32033-21cut08:10:141
308499129,2cyclictest22834-21wc12:20:001
308499129,2cyclictest22714-21ssh09:25:211
308499129,2cyclictest2116-21cut08:15:141
308499129,2cyclictest20220-21grep08:50:271
308499129,2cyclictest16861-21ssh12:35:171
308499129,2cyclictest10129-21tail08:30:201
308499128,3cyclictest22016-21ssh11:20:321
308499128,3cyclictest10379-21ssh12:11:521
308499128,2cyclictest29930-21grep09:30:151
308499126,4cyclictest28085-21diskmemload11:59:241
3084991210,1cyclictest21573-21grep08:55:191
308499120,9cyclictest0-21swapper/110:05:131
308099129,2cyclictest26852-21cut10:45:170
308099129,2cyclictest23271-21sort09:00:010
308099129,2cyclictest15996-21grep07:35:180
308099129,2cyclictest12309-21grep10:55:260
308099128,3cyclictest9677-21wc10:15:240
308099124,4cyclictest0-21swapper/012:35:130
308099123,7cyclictest7643-21sh12:30:150
308099123,5cyclictest13-21ksoftirqd/010:24:140
308999118,2cyclictest4185-21if_enp2s007:10:173
308999118,2cyclictest3404-21ssh10:30:583
308999118,2cyclictest32350-21ssh09:11:503
308999118,2cyclictest2880-21ssh10:11:363
308999118,2cyclictest20805-21cat08:55:003
308999118,2cyclictest19526-21tail11:00:163
308999118,2cyclictest18215-21ssh09:42:133
308999118,2cyclictest17802-21ssh12:35:263
308999118,2cyclictest15325-21ssh09:20:443
308999118,2cyclictest12256-21sh10:36:123
308999118,2cyclictest12256-21sh10:36:123
308999118,2cyclictest11483-21ssh11:15:123
308999118,2cyclictest11039-21sed07:25:173
308999117,3cyclictest24789-21wc09:00:223
308699119,1cyclictest5663-21cut11:30:222
308699119,1cyclictest25465-21grep07:55:152
308699118,2cyclictest9615-21ssh11:52:042
308699118,2cyclictest862-21ssh11:46:462
308699118,2cyclictest8335-21tr07:20:142
308699118,2cyclictest6525-21ssh09:55:072
308699118,2cyclictest6405-21head09:35:162
308699118,2cyclictest32175-21grep08:10:152
308699118,2cyclictest30356-21rm12:05:012
308699118,2cyclictest27825-21dump-pmu-power11:25:002
308699118,2cyclictest24369-21rm10:05:182
308699118,2cyclictest20844-21cat08:55:002
308699118,2cyclictest19421-21taskset07:41:222
308699118,2cyclictest18715-21ssh11:38:382
308699118,2cyclictest18032-21ssh11:57:082
308699118,2cyclictest13907-21cat07:30:192
308699110,10cyclictest22356-21cut09:45:142
308499118,2cyclictest7689-21cat09:55:191
308499118,2cyclictest26739-21chrt07:58:561
308499118,2cyclictest17398-21cat10:20:181
308499118,2cyclictest17083-21ssh09:22:071
308499118,2cyclictest12981-21grep08:35:261
308499118,2cyclictest12889-21grep11:15:211
308499117,3cyclictest8173-21cat08:25:251
308499117,3cyclictest25-21ksoftirqd/110:33:141
308499117,1cyclictest11708-21proc_pri07:25:241
308499116,4cyclictest5940-21unixbench_multi08:20:261
308499110,9cyclictest22521-21ntp_kernel_err11:40:211
308099118,2cyclictest831-21cut10:30:150
308099118,2cyclictest7907-21ssh09:55:210
308099118,2cyclictest7430-21tail09:35:250
308099118,2cyclictest7313-21cut08:25:160
308099118,2cyclictest4305-21sh11:30:140
308099118,2cyclictest32519-21sed12:05:260
308099118,2cyclictest31448-21ssh09:11:060
308099118,2cyclictest28441-21grep11:05:200
308099118,2cyclictest28382-21ssh11:44:260
308099118,2cyclictest27656-21grep09:05:250
308099118,2cyclictest2286-21cut10:50:150
308099118,2cyclictest20562-21rm10:03:120
308099118,2cyclictest18377-21cut07:40:160
308099118,2cyclictest15818-21tail09:40:250
308099118,2cyclictest14605-21grep11:35:270
308099118,2cyclictest1440-21ssh10:10:290
308099113,4cyclictest225-21systemd-journal08:14:550
222782110,0sleep10-21swapper/112:00:141
308999108,1cyclictest32161-21expr08:10:153
308999108,1cyclictest28732-21sed08:00:273
308999108,1cyclictest24117-21chrt07:50:443
308999108,1cyclictest17534-21sendmail_mailtr08:45:223
308999108,1cyclictest16873-21awk07:35:273
308999108,1cyclictest14282-21df08:40:153
308999108,1cyclictest11070-21chrt08:34:133
308999107,2cyclictest24715-21bc07:55:103
308999107,2cyclictest21544-21cut08:55:193
308999107,2cyclictest18812-21cut07:40:203
308699108,1cyclictest57712sleep207:15:132
308699108,1cyclictest29854-21expr08:05:162
308699108,1cyclictest25416-21taskset09:02:172
308699108,1cyclictest18561-21ssh11:00:062
308699108,1cyclictest107972sleep208:30:382
308699107,2cyclictest7569-21cat08:25:172
308699107,2cyclictest22094-21awk07:50:002
308699107,2cyclictest12680-21cat07:30:002
308499108,1cyclictest5667-21basename07:15:111
308499108,1cyclictest31506-21chrt08:08:091
308499108,1cyclictest26931-21wc09:05:181
308499108,1cyclictest209982sleep107:45:171
308499108,1cyclictest169782sleep107:35:441
308499108,1cyclictest156472sleep108:41:431
308499107,2cyclictest5655-21cut09:15:211
308499107,2cyclictest23074-21expr07:50:161
308499107,2cyclictest22105-21ssh09:45:061
308099108,1cyclictest38882sleep007:10:140
308099108,1cyclictest30485-21rm11:45:170
308099108,1cyclictest270852sleep008:00:070
308099108,1cyclictest19964-21wc08:50:260
308099108,1cyclictest14928-21cat11:55:200
308099108,1cyclictest108462sleep008:31:160
308099107,2cyclictest22642-21tr07:50:140
308099107,2cyclictest17791-21sed08:45:260
308099107,2cyclictest15646-21chrt08:41:420
308099102,5cyclictest23208-21seq10:05:050
308099100,9cyclictest25338-21grep07:55:150
30869996,2cyclictest3356-21aten2_r7power_e07:10:132
30869990,8cyclictest26547-21tune2fs09:05:152
30849984,3cyclictest16314-21grep08:45:141
30809986,1cyclictest25516-21taskset09:03:330
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional