You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-09 - 18:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Aug 09, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
111529873,8sleep00-21swapper/007:08:250
114129368,9sleep20-21swapper/207:08:442
100729166,9sleep30-21swapper/307:07:043
109628265,13sleep10-21swapper/107:08:101
207952550,0sleep00-21swapper/012:19:560
321372510,0sleep20-21swapper/211:28:162
272642290,0sleep00-21swapper/010:45:560
133899290,1cyclictest0-21swapper/210:20:142
1340992724,2cyclictest39-21ksoftirqd/309:50:003
133499261,16cyclictest225-21systemd-journal10:59:591
133199252,22cyclictest32606-21sh09:13:120
134099240,23cyclictest0-21swapper/312:17:313
134099240,0cyclictest0-21swapper/309:20:253
133499240,23cyclictest0-21swapper/109:13:301
133199240,21cyclictest0-21swapper/012:37:110
134099230,21cyclictest0-21swapper/312:26:263
133899230,1cyclictest0-21swapper/209:38:182
133199235,6cyclictest225-21systemd-journal10:59:590
1334992220,1cyclictest0-21swapper/111:01:541
1334992220,1cyclictest0-21swapper/109:27:441
134099216,12cyclictest17274-21ssh11:00:123
133899210,20cyclictest0-21swapper/212:09:012
133899210,20cyclictest0-21swapper/210:47:302
133899200,19cyclictest0-21swapper/210:10:172
1334992018,1cyclictest0-21swapper/110:40:211
1334991918,1cyclictest0-21swapper/109:48:081
133499190,19cyclictest0-21swapper/112:24:591
133499190,18cyclictest0-21swapper/112:29:231
1340991818,0cyclictest0-21swapper/311:15:313
133499181,15cyclictest0-21swapper/111:25:131
133499180,18cyclictest0-21swapper/110:01:431
133499180,16cyclictest0-21swapper/110:38:001
133499180,16cyclictest0-21swapper/109:36:281
1331991814,3cyclictest13-21ksoftirqd/010:15:120
61212170,0sleep30-21swapper/310:53:153
133899170,17cyclictest0-21swapper/209:16:542
1334991716,1cyclictest0-21swapper/111:58:011
1331991716,1cyclictest0-21swapper/011:49:100
133199170,0cyclictest0-21swapper/010:42:530
134099160,1cyclictest0-21swapper/310:15:083
134099160,15cyclictest0-21swapper/311:10:143
1334991615,1cyclictest0-21swapper/110:48:171
133499161,14cyclictest26552-21diskmemload11:15:001
133499160,2cyclictest0-21swapper/112:32:411
133499160,1cyclictest0-21swapper/110:51:101
133499160,1cyclictest0-21swapper/109:19:191
133199165,6cyclictest6372-21perf10:15:000
1331991615,1cyclictest13-21ksoftirqd/009:39:340
1340991510,3cyclictest12496-21munin-run12:15:003
1338991514,1cyclictest0-21swapper/212:33:362
133899150,1cyclictest0-21swapper/211:07:142
133899150,1cyclictest0-21swapper/210:29:452
133899150,1cyclictest0-21swapper/210:29:452
133899150,11cyclictest0-21swapper/209:21:312
133499152,10cyclictest4608-21/usr/sbin/munin12:10:251
1334991512,2cyclictest25-21ksoftirqd/109:33:531
133199151,1cyclictest141rcu_preempt09:42:000
1340991414,0cyclictest0-21swapper/311:38:583
1340991413,1cyclictest0-21swapper/310:31:103
1340991413,1cyclictest0-21swapper/310:13:023
1340991412,1cyclictest0-21swapper/309:59:283
1340991411,2cyclictest27264-21wc11:05:253
134099141,11cyclictest22587-21apache_accesses10:05:133
1340991410,3cyclictest27934-21tail09:10:183
134099140,13cyclictest16312-21grep08:45:283
133899146,3cyclictest32-21ksoftirqd/212:15:002
1338991411,2cyclictest17725-21grep08:50:182
1338991411,1cyclictest0-21swapper/211:23:262
1338991410,3cyclictest30538-21head09:30:282
1338991410,3cyclictest10931-21cat07:30:002
133899140,2cyclictest0-21swapper/209:27:032
133899140,14cyclictest0-21swapper/210:39:132
133499145,5cyclictest225-21systemd-journal08:05:001
133499140,1cyclictest0-21swapper/112:03:411
133499140,1cyclictest0-21swapper/109:43:121
133499140,14cyclictest0-21swapper/110:11:211
1331991413,1cyclictest1324-21cyclictest12:04:440
1331991410,3cyclictest14931-21tr08:45:150
1331991410,2cyclictest1705-21grep07:10:150
134099132,9cyclictest16743-21awk11:00:003
1340991313,0cyclictest0-21swapper/312:23:353
134099131,1cyclictest0-21swapper/310:25:083
134099131,1cyclictest0-21swapper/310:25:083
1340991310,3cyclictest0-21swapper/310:02:093
1340991310,2cyclictest4154-21cron12:10:003
1340991310,2cyclictest32591-21grep08:15:133
1340991310,2cyclictest13588-21wc08:40:223
134099130,2cyclictest0-21swapper/310:22:403
134099130,1cyclictest0-21swapper/309:05:143
133899139,3cyclictest16879-21wc08:50:012
133899139,3cyclictest14324-21cut12:35:142
133899134,4cyclictest28019-21idleruntime-cro12:05:012
1338991312,1cyclictest0-21swapper/207:55:132
1338991311,1cyclictest0-21swapper/210:52:072
1338991310,2cyclictest10296-21perf11:35:002
133899130,13cyclictest0-21swapper/210:56:272
133899130,12cyclictest26539-21cron09:10:002
133499135,4cyclictest23130-21munin-run10:25:011
133499134,6cyclictest13084-21apache212:07:301
1334991310,2cyclictest6369-21basename07:20:141
1334991310,2cyclictest19341-21ssh11:20:191
1334991310,2cyclictest18778-21grep08:50:281
1334991310,2cyclictest11753-21cut09:20:151
133499130,1cyclictest0-21swapper/111:40:221
133499130,12cyclictest28662-21grep11:45:181
133199139,3cyclictest11063-21sendmail09:20:010
133199134,8cyclictest13280-21perf07:35:000
133199134,5cyclictest225-21systemd-journal09:30:000
133199132,6cyclictest2246-21cron08:20:000
133199131,6cyclictest14445-21munin-run08:45:000
1331991310,2cyclictest8273-21grep10:35:150
1331991310,2cyclictest6069-21latency_hist07:20:000
1331991310,2cyclictest4664-21basename08:24:590
1331991310,2cyclictest4363-21sed11:30:270
1331991310,2cyclictest27650-21wc11:25:200
1331991310,2cyclictest26386-21grep11:05:170
1331991310,2cyclictest23109-21cron10:25:000
1331991310,2cyclictest19729-21grep08:55:130
1331991310,2cyclictest12493-21munin-plugin-st12:15:000
133199130,1cyclictest0-21swapper/009:55:460
133199130,10cyclictest22615-21sh10:05:130
134099129,2cyclictest6470-21cut07:20:143
134099129,2cyclictest5068-21ssh11:50:263
134099129,2cyclictest29412-21grep09:30:183
134099129,2cyclictest18087-21cat10:40:263
134099125,5cyclictest26552-21diskmemload09:25:143
134099122,6cyclictest0-21swapper/311:30:133
1340991210,1cyclictest2795-21grep08:20:143
1340991210,1cyclictest27735-21grep11:45:133
134099120,11cyclictest27343-21head11:25:173
133899129,2cyclictest5858-21tr09:55:152
133899129,2cyclictest32303-21ls08:15:002
133899129,2cyclictest30772-21grep12:25:192
133899129,2cyclictest26665-21grep08:00:232
133899129,2cyclictest2386-21wc08:20:002
133899129,2cyclictest15822-21chrt10:00:242
133899129,2cyclictest11706-21sed11:15:272
1338991211,1cyclictest0-21swapper/211:49:562
133899120,12cyclictest0-21swapper/211:35:152
133899120,11cyclictest0-21swapper/209:50:472
133499129,2cyclictest4223-21grep11:50:191
133499129,2cyclictest23414-21grep10:05:171
133499129,2cyclictest20504-21cat07:50:011
133499129,2cyclictest20504-21cat07:50:001
133499129,2cyclictest20367-21grep08:55:161
133499129,2cyclictest17612-21grep07:40:281
133499129,2cyclictest13657-21dpkg10:00:011
133499129,2cyclictest11160-21sed11:15:221
133199129,2cyclictest8949-21tr07:25:130
133199129,2cyclictest32175-21grep10:30:160
133199129,2cyclictest26656-21cat09:10:000
133199129,2cyclictest2458-21sh10:50:260
133199129,2cyclictest2323-21grep11:10:180
133199129,2cyclictest22762-21cron07:55:000
133199129,2cyclictest20256-21chrt11:20:290
133199129,2cyclictest17335-21tr08:50:150
133199129,2cyclictest12456-21grep11:55:150
133199129,2cyclictest11752-21grep09:20:150
133199128,2cyclictest30970-21head08:10:180
133199124,7cyclictest18025-21idleruntime-cro07:45:000
1331991212,0cyclictest0-21swapper/012:06:160
1331991210,1cyclictest22210-21grep12:20:160
134099119,1cyclictest537-21snmpd12:34:413
134099119,1cyclictest27806-21chrt08:05:143
134099119,1cyclictest24983-21grep10:45:153
134099119,1cyclictest18618-21cat08:50:273
134099118,2cyclictest8976-21sed08:30:273
134099118,2cyclictest4005-21basename07:15:143
134099118,2cyclictest21637-21sh11:40:503
134099118,2cyclictest21375-21cat12:00:223
134099118,2cyclictest20713-21latency_hist09:45:003
134099118,2cyclictest18017-21cron07:45:003
134099118,2cyclictest1795-21grep07:10:163
134099118,2cyclictest15642-21grep07:40:003
134099118,2cyclictest12778-21sed07:30:303
134099118,2cyclictest12103-21expr11:55:133
1340991110,1cyclictest0-21swapper/312:35:113
1340991110,1cyclictest0-21swapper/309:35:133
134099110,11cyclictest0-21swapper/310:35:143
134099110,10cyclictest7223-21ssh09:17:003
133899119,1cyclictest12940-21grep11:55:182
133899118,2cyclictest8858-21grep10:15:272
133899118,2cyclictest7515-21cut08:30:152
133899118,2cyclictest5674-21sed08:25:182
133899118,2cyclictest3756-21ls08:20:222
133899118,2cyclictest3341-21ssh11:50:122
133899118,2cyclictest3058-21cat07:10:272
133899118,2cyclictest29449-21grep08:05:292
133899118,2cyclictest2784-21ssh11:10:212
133899118,2cyclictest27616-21ssh10:07:482
133899118,2cyclictest24773-21ssh09:46:392
133899118,2cyclictest21458-21grep12:20:132
133899118,2cyclictest20566-21interrupts08:55:182
133899118,2cyclictest19069-21ssh11:40:102
133899118,2cyclictest19052-21sed07:45:182
133899118,2cyclictest19052-21sed07:45:172
133899118,2cyclictest16370-21ssh10:40:132
133899118,2cyclictest15367-21ssh12:15:412
133899118,2cyclictest14425-21cron08:45:002
133899118,2cyclictest13783-21tail09:40:202
133899118,2cyclictest13326-21wc07:35:012
133899113,6cyclictest1235-21ntpd11:00:192
133899113,5cyclictest16055-21cut07:40:152
133899110,4cyclictest0-21swapper/210:32:142
133899110,1cyclictest141rcu_preempt09:10:142
133499118,2cyclictest7594-21ssh11:32:551
133499118,2cyclictest7360-21grep08:30:151
133499118,2cyclictest4346-21chrt08:21:151
133499118,2cyclictest31275-21tr08:10:211
133499118,2cyclictest25542-21grep09:05:211
133499118,2cyclictest23919-21grep10:25:131
133499118,2cyclictest23919-21grep10:25:131
133499118,2cyclictest22658-21grep09:00:191
133499118,2cyclictest16746-21seq08:49:221
133499118,2cyclictest1497-21ssh10:30:281
133499118,2cyclictest14296-21taskset08:43:241
133499118,2cyclictest13174-21cut12:15:151
133499118,2cyclictest11879-21wc11:35:201
133499118,2cyclictest10816-21cron07:30:001
133499117,3cyclictest3137-21wc07:10:271
133499114,3cyclictest225-21systemd-journal07:16:431
1334991110,1cyclictest0-21swapper/112:39:251
133499110,10cyclictest32605-21cut08:15:141
133499110,10cyclictest12140-21sed08:40:001
133499110,10cyclictest11110-21sh10:17:111
133199118,2cyclictest984-21ssh09:32:410
133199118,2cyclictest7499-21ssh12:30:250
133199118,2cyclictest3303-21ssh11:50:110
133199118,2cyclictest30732-21grep12:25:180
133199118,2cyclictest29366-21ssh09:50:050
133199118,2cyclictest25481-21cut08:00:150
133199118,2cyclictest24361-21ssh09:46:200
133199118,2cyclictest23448-21ssh10:25:060
133199118,2cyclictest23448-21ssh10:25:060
133199118,2cyclictest22360-21ssh11:41:240
133199118,2cyclictest18324-21grep07:45:140
133199118,2cyclictest18324-21grep07:45:140
133199118,2cyclictest16081-21ssh11:18:460
133199118,2cyclictest14462-21ssh11:36:440
1331991111,0cyclictest0-21swapper/008:35:030
133199110,7cyclictest225-21systemd-journal10:00:010
133199110,10cyclictest19825-21sh11:00:530
106792110,6sleep339-21ksoftirqd/308:35:193
134099108,1cyclictest30754-21df_inode08:10:153
134099108,1cyclictest25915-21sh08:00:163
134099108,1cyclictest18488-21grep07:45:153
134099108,1cyclictest18488-21grep07:45:153
134099108,1cyclictest0-21swapper/311:20:133
134099107,2cyclictest5037-21gpgv09:55:003
134099107,2cyclictest23066-21cut07:55:143
134099107,2cyclictest10323-21grep07:25:253
134099100,9cyclictest22404-21expr09:00:163
134099100,10cyclictest6847-21taskset08:27:533
133899108,1cyclictest22493-21taskset07:51:352
133899108,1cyclictest15534-21chrt07:38:562
133899107,2cyclictest6576-21grep07:20:142
133899107,2cyclictest5561-21sed07:15:272
133499108,1cyclictest11131-21tr07:30:161
133499107,2cyclictest5656-21grep08:25:181
133499107,2cyclictest29734-21grep09:50:151
133499107,2cyclictest28562-21cut08:05:191
133499107,2cyclictest23698-21cut07:55:171
133499107,2cyclictest20759-21hostname07:50:141
133499100,9cyclictest28116-21sh11:05:511
133199107,2cyclictest7548-21grep08:30:150
133199107,2cyclictest6241-21fgrep08:25:240
133199107,2cyclictest28745-21grep08:05:200
133199107,2cyclictest24939-21taskset07:57:310
133199107,2cyclictest23418-21mailstats09:00:240
133199100,9cyclictest6782-21expr07:20:150
13409998,1cyclictest19934-21expr08:55:143
13409997,1cyclictest20618-21taskset07:50:033
13389997,1cyclictest23216-21ntp_states09:00:232
13389997,1cyclictest10796-21uname08:35:192
13319997,1cyclictest14205-21cut07:35:160
13349985,2cyclictest14423-21cat07:35:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional