You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-27 - 14:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Mon Jul 27, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2018929179,8sleep20-21swapper/207:08:002
1797629176,10sleep00-21swapper/007:05:000
2014628364,14sleep10-21swapper/107:07:261
1821928168,9sleep30-21swapper/307:05:123
97112720,0sleep20-21swapper/211:03:252
2043299250,25cyclictest0-21swapper/211:17:452
2043099250,1cyclictest0-21swapper/112:01:191
2043699242,1cyclictest0-21swapper/310:54:303
2043299240,24cyclictest0-21swapper/209:44:472
2043299240,24cyclictest0-21swapper/209:44:472
2043699231,1cyclictest0-21swapper/307:40:143
2043299230,23cyclictest0-21swapper/210:26:412
2043099232,20cyclictest26451-21perf10:35:001
2043099230,4cyclictest0-21swapper/111:29:381
2043099230,4cyclictest0-21swapper/111:29:371
2043099230,2cyclictest0-21swapper/111:19:391
20432992222,0cyclictest0-21swapper/210:02:052
2043299220,22cyclictest0-21swapper/211:50:052
2043099221,1cyclictest0-21swapper/109:13:371
2043099220,1cyclictest0-21swapper/109:15:331
20426992221,1cyclictest0-21swapper/010:38:420
114132220,0sleep10-21swapper/109:46:451
20430992120,1cyclictest0-21swapper/109:51:561
20430992020,0cyclictest0-21swapper/110:42:211
20426992016,1cyclictest0-21swapper/011:10:460
20426992014,2cyclictest0-21swapper/011:24:380
2043699191,2cyclictest0-21swapper/310:34:263
20432991917,1cyclictest32-21ksoftirqd/210:23:082
2043299191,1cyclictest141rcu_preempt09:47:232
2043099190,1cyclictest0-21swapper/110:04:201
20436991813,3cyclictest6133-21idleruntime-cro09:25:003
2043699180,16cyclictest0-21swapper/311:48:553
2043099180,18cyclictest0-21swapper/112:32:071
9002172,3sleep00-21swapper/009:40:250
9002172,3sleep00-21swapper/009:40:240
20436991715,2cyclictest0-21swapper/312:02:053
2043699171,15cyclictest0-21swapper/309:25:133
20432991713,2cyclictest32-21ksoftirqd/210:53:162
2043299170,2cyclictest0-21swapper/212:20:262
2043299170,16cyclictest0-21swapper/212:28:252
20430991711,5cyclictest17476-21ssh12:05:291
2043099170,1cyclictest0-21swapper/111:30:221
2043099170,0cyclictest0-21swapper/111:45:301
2042699170,1cyclictest0-21swapper/010:50:520
2043299160,1cyclictest141rcu_preempt10:58:532
2043099168,7cyclictest3439-21timerandwakeup10:20:241
2043099166,6cyclictest225-21systemd-journal10:55:001
2043099163,5cyclictest1796-21kworker/1:112:10:471
2043099160,1cyclictest0-21swapper/112:20:161
2043099160,16cyclictest0-21swapper/109:59:261
2042699169,6cyclictest14021-21munin-node11:40:450
20426991615,1cyclictest13-21ksoftirqd/009:10:160
2042699160,2cyclictest0-21swapper/009:52:310
20436991514,1cyclictest0-21swapper/311:31:023
2043699151,0cyclictest0-21swapper/309:53:423
20432991513,1cyclictest32-21ksoftirqd/209:55:202
20432991512,2cyclictest32-21ksoftirqd/212:02:072
20432991512,1cyclictest141rcu_preempt10:44:092
2043299150,14cyclictest0-21swapper/211:38:402
2043099152,1cyclictest4871-21basename11:20:121
20430991512,2cyclictest0-21swapper/111:03:481
2042699150,14cyclictest0-21swapper/012:22:000
3462140,0sleep00-21swapper/011:16:530
20436991413,1cyclictest10744-21kworker/3:011:20:523
20436991412,1cyclictest0-21swapper/312:12:413
20436991411,2cyclictest27037-21ssh10:15:213
20436991411,2cyclictest18388-21chrt12:06:133
2043699140,8cyclictest10076-21/usr/sbin/munin10:25:203
2043299149,2cyclictest32-21ksoftirqd/211:08:542
2043299142,12cyclictest13226-21diskmemload10:32:532
20432991411,3cyclictest0-21swapper/209:13:042
20432991411,2cyclictest7916-21grep11:40:262
20432991411,2cyclictest32-21ksoftirqd/211:48:372
2043299140,13cyclictest32-21ksoftirqd/209:30:152
2043099148,5cyclictest1151-21tr12:35:151
2043099146,6cyclictest24012-21/usr/sbin/munin11:50:231
2043099140,12cyclictest0-21swapper/110:05:101
2042699149,3cyclictest17733-21tr10:10:160
2042699140,1cyclictest0-21swapper/010:05:520
2042699140,13cyclictest18981-21grep10:30:150
20436991312,1cyclictest10744-21kworker/3:011:36:233
20436991312,1cyclictest10744-21kworker/3:011:02:133
20436991310,2cyclictest9675-21sh07:55:013
20436991310,2cyclictest952-21wget12:35:143
20436991310,2cyclictest6068-21cron08:54:593
20436991310,2cyclictest32504-21cron07:35:003
20436991310,2cyclictest32063-21cut09:40:153
20436991310,2cyclictest32063-21cut09:40:153
20436991310,2cyclictest22769-21sh07:15:003
20436991310,2cyclictest18263-21head10:10:193
20436991310,2cyclictest1119-21cut12:15:233
20436991310,2cyclictest10643-21grep07:55:173
2043699130,13cyclictest0-21swapper/310:22:593
2043699130,12cyclictest0-21swapper/309:33:513
2043299135,4cyclictest225-21systemd-journal08:40:002
20432991310,2cyclictest9897-21tail10:05:182
20432991310,2cyclictest32686-21grep08:40:222
20432991310,2cyclictest32-21ksoftirqd/211:55:202
20432991310,2cyclictest23083-21grep11:30:222
20432991310,2cyclictest16620-21grep08:05:262
20432991310,2cyclictest13447-21wget11:25:162
20432991310,2cyclictest13447-21wget11:25:162
20432991310,2cyclictest12562-21grep08:00:142
20432991310,2cyclictest1104-21grep07:35:192
2043299130,2cyclictest0-21swapper/212:06:322
2043299130,13cyclictest0-21swapper/212:30:202
2043299130,12cyclictest22009-21cut11:10:212
2043099133,1cyclictest225-21systemd-journal11:44:371
2043099132,8cyclictest38150irq/131-snd_hda08:40:171
2043099131,9cyclictest38150irq/131-snd_hda12:25:131
2043099131,11cyclictest15808-21ssh10:47:281
20430991310,2cyclictest7284-21cron07:50:011
20430991310,2cyclictest31949-21grep11:55:191
20430991310,2cyclictest26543-21cron08:29:591
2043099130,9cyclictest0-21swapper/109:05:141
2043099130,12cyclictest22150-21cat11:10:221
2042699139,2cyclictest32709-21tr08:40:210
2042699139,2cyclictest24575-21cut09:55:150
2042699139,2cyclictest21690-21cat07:10:200
2042699139,2cyclictest10002-21grep07:55:140
2042699139,2cyclictest0-21swapper/010:05:000
20426991310,2cyclictest7311-21sort09:25:170
20426991310,2cyclictest432-21basename07:35:160
20426991310,2cyclictest30437-21grep07:30:150
20426991310,2cyclictest14588-21grep11:25:220
20426991310,2cyclictest14588-21grep11:25:210
20426991310,2cyclictest14200-21sed11:05:260
20426991310,2cyclictest11381-21tr09:05:150
2042699130,2cyclictest0-21swapper/011:55:280
2042699130,11cyclictest0-21swapper/010:47:510
2042699130,0cyclictest0-21swapper/010:19:260
2043699129,2cyclictest8593-21sed09:45:183
2043699129,2cyclictest31197-21sh09:39:593
2043699129,2cyclictest28758-21ssh09:57:153
2043699129,2cyclictest24675-21grep08:25:143
2043699129,2cyclictest23326-21wc08:20:213
2043699129,2cyclictest14484-21cat08:05:003
2043699128,3cyclictest3656-21head10:40:183
2043699128,3cyclictest31411-21tr11:55:183
2043699128,3cyclictest21653-21ssh12:27:583
2043699127,3cyclictest3482-21wc10:59:593
2043699126,4cyclictest797-21sh11:17:133
2043699126,2cyclictest225-21systemd-journal08:15:003
2043699123,6cyclictest25155-21cron07:20:003
2043699120,5cyclictest0-21swapper/311:25:113
2043699120,5cyclictest0-21swapper/311:25:113
2043699120,11cyclictest0-21swapper/312:23:383
2043299129,2cyclictest7720-21wc09:25:182
2043299129,2cyclictest32-21ksoftirqd/209:20:162
2043299129,2cyclictest10531-21cut07:55:162
2043299128,3cyclictest7292-21dump-pmu-power07:50:012
2043299124,5cyclictest23775-21diskmemload12:10:152
2043299121,3cyclictest0-21swapper/209:35:132
2043299120,2cyclictest537-21snmpd10:15:192
2043299120,2cyclictest0-21swapper/212:19:102
2043099129,2cyclictest32570-21tr09:40:181
2043099129,2cyclictest32570-21tr09:40:171
2043099129,2cyclictest29298-21tr08:35:151
2043099129,2cyclictest2649-21cut08:45:231
2043099128,3cyclictest23797-21grep07:15:181
2043099128,3cyclictest11584-21sed07:55:261
2043099128,2cyclictest7444-21grep08:55:211
2043099128,2cyclictest459-21grep07:35:161
2043099124,6cyclictest11889-1kworker/u9:012:16:431
2043099120,6cyclictest21690-21kworker/1:008:35:001
2042699129,2cyclictest9978-21tr09:00:240
2042699129,2cyclictest7907-21sed12:00:170
2042699129,2cyclictest31938-21grep11:35:270
2042699129,2cyclictest3021-21cat08:45:290
2042699129,2cyclictest29746-21wc10:55:270
2042699129,2cyclictest23785-21head08:20:260
2042699129,2cyclictest15619-21cut11:45:200
2042699129,2cyclictest15086-21tr09:30:150
2042699129,2cyclictest12430-21grep08:00:140
2042699128,3cyclictest11509-21ls10:25:230
2042699124,7cyclictest28944-21awk08:35:000
2042699123,5cyclictest25742-21users12:10:280
2042699121,5cyclictest15590-21perf09:50:010
2042699120,0cyclictest0-21swapper/012:34:050
2043699119,1cyclictest24670-21ssh12:30:103
2043699118,2cyclictest7294-21cut11:40:203
2043699118,2cyclictest6892-21ssh10:03:553
2043699118,2cyclictest6446-21wc07:45:253
2043699118,2cyclictest30040-21ssh10:36:173
2043699118,2cyclictest28534-21head07:25:163
2043699118,2cyclictest27196-21ssh11:52:223
2043699118,2cyclictest2484-21cut08:45:223
2043699118,2cyclictest22065-21basename09:15:113
2043699118,2cyclictest21059-21sensors08:15:223
2043699118,2cyclictest20546-21ssh11:10:103
2043699118,2cyclictest17992-21ssh09:12:123
2043699118,2cyclictest15681-21sed08:05:173
2043699118,2cyclictest13105-21hddtemp_smartct11:05:173
2043699118,2cyclictest11587-21ssh10:05:543
2043699118,2cyclictest11316-21wget10:45:133
2043699113,4cyclictest225-21systemd-journal08:38:573
2043299118,2cyclictest4509-21ssh12:36:432
2043299118,2cyclictest3234-21sed08:45:302
2043299118,2cyclictest2745-21apache_accesses07:40:122
2043299118,2cyclictest23478-21ssh09:15:212
2043299118,2cyclictest23321-21tr07:15:152
2043299118,2cyclictest17895-21sort08:10:182
2043299118,2cyclictest12346-21head09:05:232
2043299118,2cyclictest11886-21ssh10:45:162
2043299117,3cyclictest30091-21cat07:30:002
2043299117,3cyclictest18281-21sed10:10:182
2043299117,3cyclictest17194-21wc09:50:182
2043299110,11cyclictest0-21swapper/210:36:062
2043099119,1cyclictest31922-21grep09:20:201
2043099118,2cyclictest8167-21systemctl09:25:221
2043099118,2cyclictest32305-21ssh11:35:411
2043099118,2cyclictest29937-21sleep07:29:591
2043099118,2cyclictest28556-21ssh10:55:171
2043099118,2cyclictest24065-21grep09:35:171
2043099118,2cyclictest22985-21sed07:15:001
2043099118,2cyclictest19385-21ssh09:32:191
2043099118,2cyclictest18011-21ssh10:10:161
2043099118,2cyclictest10126-21ssh10:25:141
2043099117,3cyclictest32192-21chrt10:37:581
2043099117,3cyclictest21640-21taskset08:18:201
2043099113,8cyclictest38150irq/131-snd_hda10:18:451
2042699118,2cyclictest6982-21ssh10:42:040
2042699118,2cyclictest6183-21ssh12:38:000
2042699118,2cyclictest5967-21taskset08:53:450
2042699118,2cyclictest3974-21grep07:40:210
2042699118,2cyclictest3640-21ssh11:00:030
2042699118,2cyclictest32495-21wc09:20:270
2042699118,2cyclictest2546-21ssh12:16:100
2042699118,2cyclictest24664-21sh09:35:210
2042699118,2cyclictest23996-21grep11:50:220
2042699118,2cyclictest23408-21expr07:15:150
2042699118,2cyclictest23061-21basename09:15:190
2042699118,2cyclictest22070-21expr11:30:150
2042699118,2cyclictest20996-21sendmail_mailst08:15:200
2042699118,2cyclictest1917-21grep10:20:130
2042699118,2cyclictest17381-21cut08:10:160
2042699118,2cyclictest16126-21ssh12:05:170
2042699118,2cyclictest15344-21cat08:05:140
2042699114,1cyclictest0-21swapper/007:25:130
2042699110,10cyclictest0-21swapper/012:25:160
2043699108,1cyclictest6773-21expr08:55:163
2043699108,1cyclictest31910-21cat08:40:173
2043699107,2cyclictest27105-21expr08:30:153
2043699107,2cyclictest1448-21tr07:35:203
2043299108,1cyclictest6076-21ssh11:20:202
2043299108,1cyclictest4526-21tune2fs08:50:162
2043299108,1cyclictest25758-21gzip08:25:222
2043299108,1cyclictest25443-21apache_volume07:20:122
2043299107,2cyclictest7783-21grep07:50:122
2043299107,2cyclictest7131-21tail08:55:182
2043299107,2cyclictest27524-21grep08:30:172
2043299107,2cyclictest20983-21expr07:10:152
2043299107,2cyclictest20290-21sort08:15:162
2043299100,5cyclictest22355-21cpuspeed_turbos08:20:142
2043099108,1cyclictest45242sleep107:40:511
2043099108,1cyclictest32335-21taskset07:32:541
2043099108,1cyclictest13160-21basename08:00:191
2043099108,1cyclictest12100-21ssh11:05:081
2043099107,2cyclictest9878-21grep09:00:231
2043099107,2cyclictest3944-21grep08:50:131
2043099107,2cyclictest16737-21taskset08:06:261
2043099104,5cyclictest9737-21wc07:55:001
2043099100,9cyclictest27280-21chrt07:21:411
2043099100,9cyclictest17480-21egrep08:10:161
2042699108,1cyclictest78492sleep007:50:120
2042699108,1cyclictest7278-21uname08:55:200
2042699108,1cyclictest6152-21cat07:45:220
2042699108,1cyclictest26991-21cat07:20:260
2042699108,1cyclictest24468-21aten2_r7power_c08:25:140
2042699107,2cyclictest29612-21cat08:35:160
204369997,1cyclictest8726-21cat09:00:133
204369997,1cyclictest25673-21expr07:20:143
204369997,1cyclictest11401-21expr09:05:163
204329997,1cyclictest8739-21chrt09:00:152
204329997,1cyclictest322452sleep207:31:442
204309997,1cyclictest220392sleep108:20:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional