You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-02 - 20:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Sat May 02, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1174229949,44sleep20-21swapper/207:05:242
1109729884,9sleep00-21swapper/007:05:180
1235228666,9sleep30-21swapper/307:06:543
1228028467,13sleep10-21swapper/107:05:551
12686993129,1cyclictest6118-21diskmemload12:24:253
1268699290,1cyclictest0-21swapper/311:30:133
1267999270,23cyclictest0-21swapper/111:33:181
1267999250,0cyclictest0-21swapper/111:58:391
1267999240,24cyclictest0-21swapper/110:55:471
1267999230,2cyclictest0-21swapper/107:20:001
1267299230,23cyclictest0-21swapper/011:58:460
12679992219,2cyclictest94-21kswapd009:36:521
12679992219,2cyclictest5007-21cut10:45:231
1267999220,2cyclictest0-21swapper/112:01:511
12672992220,1cyclictest75592sleep012:24:190
12686992119,1cyclictest24460-21cat11:55:183
12682992119,1cyclictest14208-21cut11:10:202
12682992118,2cyclictest11044-21aten2_r7power_p10:30:142
12679992119,1cyclictest8399-21fschecks_count09:30:201
12679992119,1cyclictest15200-21aten2_r7power_c09:15:121
1267999211,13cyclictest0-21swapper/109:55:131
12672992119,1cyclictest9223-21df_abs09:50:150
1267299210,20cyclictest0-21swapper/009:13:440
103472210,1sleep210351-21latency_hist10:30:002
12686992019,1cyclictest0-21swapper/312:17:283
12686992019,1cyclictest0-21swapper/309:40:193
12682992017,2cyclictest13955-21cut11:30:132
1267999201,2cyclictest17680-21kworker/1:212:22:031
1267299200,20cyclictest0-21swapper/011:42:470
39842190,2sleep20-21swapper/210:05:532
12686991918,1cyclictest0-21swapper/311:46:543
12682991917,1cyclictest11967-21cut10:50:142
1268299190,18cyclictest0-21swapper/209:18:252
12679991918,1cyclictest0-21swapper/110:32:111
12679991916,2cyclictest3182-21tail10:25:181
1267999190,1cyclictest6118-21diskmemload10:23:361
12672991917,1cyclictest3121-21perl10:25:170
12686991814,3cyclictest3425-21kworker/3:009:30:333
1268299183,6cyclictest0-21swapper/210:00:132
1268299183,3cyclictest0-21swapper/210:57:332
1267999180,18cyclictest0-21swapper/110:38:061
1267999180,17cyclictest0-21swapper/109:20:531
12672991818,0cyclictest0-21swapper/011:12:370
12672991817,1cyclictest0-21swapper/010:13:390
12672991814,4cyclictest5060-21apache211:30:300
12682991714,2cyclictest6514-21aten2_r7power_c11:45:142
12682991714,2cyclictest4619-21aten2_r7power_v11:05:152
1268299171,1cyclictest141rcu_preempt10:42:272
1268299170,1cyclictest28535-21ssh11:19:392
12679991716,1cyclictest0-21swapper/111:39:061
1267999170,1cyclictest0-21swapper/112:26:011
1267999170,16cyclictest0-21swapper/111:15:381
1267299175,10cyclictest6118-21diskmemload11:50:000
1268699162,9cyclictest0-21swapper/309:15:133
1268699161,14cyclictest0-21swapper/308:35:133
1268699160,3cyclictest0-21swapper/311:14:243
1268299162,8cyclictest0-21swapper/211:55:132
12679991616,0cyclictest0-21swapper/111:51:351
12679991614,1cyclictest0-21swapper/111:43:351
12679991613,2cyclictest0-21swapper/107:20:181
12672991614,1cyclictest215092sleep009:38:140
12686991514,1cyclictest0-21swapper/312:25:073
12686991512,2cyclictest23715-21tr09:20:153
1268299153,11cyclictest23743-21sensors11:35:222
1267999159,5cyclictest24400-21taskset10:18:501
1267999150,1cyclictest0-21swapper/107:35:141
1267999150,15cyclictest0-21swapper/111:47:581
1267299156,8cyclictest8125-21munin-run09:50:000
12672991513,1cyclictest1565-21aten2_r7power_p10:05:140
1268699146,4cyclictest0-21swapper/311:21:513
12686991413,1cyclictest0-21swapper/308:05:163
12686991411,2cyclictest28362-21grep10:20:253
12686991410,3cyclictest30483-21grep11:00:263
12686991410,3cyclictest24747-21cron07:35:003
1268299148,5cyclictest7343-21ssh12:05:002
1268299146,7cyclictest21878-21ssh12:13:152
1268299141,4cyclictest0-21swapper/209:54:532
12682991411,2cyclictest0-21swapper/212:21:212
12682991410,3cyclictest18080-21grep08:25:262
12682991410,3cyclictest15456-21cron09:35:002
1267999147,6cyclictest0-21swapper/111:28:411
1267999142,10cyclictest38150irq/131-snd_hda08:50:131
1267999141,12cyclictest6118-21diskmemload10:40:261
1267999141,11cyclictest31051-21wc07:45:211
12679991410,3cyclictest29072-21grep07:40:271
12679991410,2cyclictest5181-21sort11:25:001
1267999140,1cyclictest0-21swapper/111:11:001
1267299149,2cyclictest13-21ksoftirqd/010:55:120
1267299145,8cyclictest12668-21cyclictest10:18:540
1267299144,6cyclictest5060-21apache210:34:120
1267299141,9cyclictest22934-21idleruntime-cro09:20:000
12672991412,2cyclictest13-21ksoftirqd/010:35:130
12672991410,3cyclictest25134-21idleruntime-cro12:35:010
1267299140,2cyclictest0-21swapper/009:59:360
1268699139,3cyclictest32001-21ssh12:00:153
1268699139,3cyclictest22158-21kworker/3:111:54:233
1268699133,3cyclictest1075-21ls09:25:263
12686991312,1cyclictest3425-21kworker/3:010:07:533
12686991310,2cyclictest29465-21sort08:50:173
12686991310,2cyclictest26081-21cat10:20:013
12686991310,2cyclictest21924-21grep07:25:263
12686991310,2cyclictest17926-21grep09:35:273
12686991310,2cyclictest13900-21wc07:10:193
12686991310,2cyclictest1069-21turbostat08:59:593
1268699130,2cyclictest0-21swapper/312:14:463
1268299132,8cyclictest32167-21apt-get07:50:132
1268299131,10cyclictest55750irq/128-enp2s0-10:12:582
12682991310,2cyclictest6978-21grep09:10:152
12682991310,2cyclictest4219-21cron08:00:002
12682991310,2cyclictest25884-21sendmail10:20:012
1267999139,3cyclictest23151-21head08:35:261
1267999139,2cyclictest20499-21tr07:25:151
1267999139,2cyclictest16456-21tr12:10:141
12679991313,0cyclictest0-21swapper/109:50:131
12679991312,0cyclictest0-21swapper/110:54:481
12679991311,1cyclictest0-21swapper/112:35:271
12679991311,1cyclictest0-21swapper/112:35:271
12679991310,2cyclictest7302-21chrt09:10:171
12679991310,2cyclictest32231-21grep07:50:131
12679991310,2cyclictest30191-21wc11:00:251
12679991310,2cyclictest19182-21grep12:30:261
12679991310,2cyclictest15206-21grep08:20:191
1267999130,12cyclictest15062-21sort07:15:001
1267999130,12cyclictest12344-21grep08:15:161
1267299135,7cyclictest27481-21/usr/sbin/munin10:40:250
1267299134,4cyclictest30005-21perf11:40:000
12672991311,2cyclictest13-21ksoftirqd/011:06:280
12672991310,2cyclictest3918-21cron11:05:000
12672991310,2cyclictest24183-21fschecks_time09:20:170
1268699129,2cyclictest902-21sort09:45:163
1268699129,2cyclictest7397-21sed11:25:263
1268699129,2cyclictest424-21grep07:50:163
1268699129,2cyclictest30912-21tr07:45:203
1268699129,2cyclictest3074-21cut10:25:173
1268699129,2cyclictest22459-21grep11:15:213
1268699129,2cyclictest20867-21grep10:55:153
1268699129,2cyclictest15771-21apt12:10:003
1268699129,2cyclictest12350-21cat10:50:153
1268699129,2cyclictest11707-21ssh09:51:193
1268699129,2cyclictest1054-21ls12:40:003
1268699129,2cyclictest1054-21ls12:40:003
1268699128,3cyclictest4490-21df09:05:163
1268699127,3cyclictest0-21swapper/310:44:383
1268699121,8cyclictest39-21ksoftirqd/310:49:413
1268699121,10cyclictest0-21swapper/311:39:043
1268699120,1cyclictest0-21swapper/311:40:063
1268699120,11cyclictest17215-21head08:25:173
1268699120,0cyclictest0-21swapper/312:34:453
1268299129,2cyclictest6391-21sed11:25:172
1268299129,2cyclictest4727-21grep09:05:182
1268299129,2cyclictest30444-21grep11:00:262
1268299129,2cyclictest30008-21grep08:50:212
1268299129,2cyclictest29022-21grep07:40:272
1268299129,2cyclictest26469-21grep12:35:182
1268299129,2cyclictest26469-21grep12:35:182
1268299129,2cyclictest23787-21sh09:40:002
1268299129,2cyclictest18552-21sort07:20:182
1268299129,2cyclictest16200-21cron08:25:002
1268299122,1cyclictest6118-21diskmemload12:25:272
12682991210,1cyclictest0-21swapper/212:17:102
1268299120,12cyclictest0-21swapper/211:42:262
1268299120,11cyclictest0-21swapper/212:08:582
1267999129,2cyclictest25432-21grep12:15:161
1267999129,2cyclictest24368-21head07:30:271
1267999129,2cyclictest2306-21cut10:05:181
1267999129,2cyclictest20697-21grep08:30:271
1267999129,2cyclictest1855-21tr09:00:151
1267999129,2cyclictest12680-21seq12:07:351
1267999129,2cyclictest10645-21fgrep08:10:231
1267999128,3cyclictest25347-21uname09:40:191
1267999121,9cyclictest487-21avahi-daemon08:05:151
1267299129,2cyclictest8763-21head12:05:180
1267299129,2cyclictest7020-21grep08:05:160
1267299129,2cyclictest5263-21wc09:05:230
1267299129,2cyclictest3270-21head09:00:270
1267299129,2cyclictest28546-21taskset12:16:570
1267299129,2cyclictest26677-21grep10:20:130
1267299129,2cyclictest12400-21grep08:15:160
1267299129,2cyclictest11802-21chrt11:28:480
1267299125,6cyclictest5047-21apache212:40:000
1267299125,6cyclictest5047-21apache212:40:000
1267299124,4cyclictest225-21systemd-journal11:55:000
1267299123,5cyclictest155052sleep007:15:130
1267299122,4cyclictest0-21swapper/012:25:210
12672991210,1cyclictest32482-21head08:55:190
12672991210,1cyclictest24096-21grep08:40:140
1267299120,1cyclictest0-21swapper/009:32:590
1268699118,2cyclictest8842-21ssh11:07:163
1268699118,2cyclictest3355-21chrt09:01:013
1268699118,2cyclictest30449-21ssh10:02:553
1268699118,2cyclictest21195-21grep10:35:263
1268699118,2cyclictest19213-21sh09:55:383
1268699118,2cyclictest16633-21ntpq07:15:203
1268699118,2cyclictest16126-21taskset08:24:073
1268699118,2cyclictest13890-21ssh10:12:083
1268699118,2cyclictest11745-21cut08:15:143
1268699118,2cyclictest11151-21cpu10:30:153
1268699117,3cyclictest20744-21grep08:30:263
1268699110,10cyclictest0-21swapper/309:10:533
1268299118,2cyclictest32684-21wget09:45:132
1268299118,2cyclictest32580-21head09:25:182
1268299118,2cyclictest31811-21taskset07:49:042
1268299118,2cyclictest31036-21ssh11:20:222
1268299118,2cyclictest30885-21ssh10:22:222
1268299118,2cyclictest25757-21chrt08:42:542
1268299118,2cyclictest24714-21expr09:40:152
1268299118,2cyclictest17225-21grep09:55:142
1268299118,2cyclictest15509-21cat11:50:152
1268299118,2cyclictest13967-21wc07:10:202
1268299118,2cyclictest11360-21sleep10:50:002
1268299117,3cyclictest13806-21cron08:20:012
1268299115,6cyclictest5047-21apache209:23:452
1268299110,10cyclictest22260-21ssh12:32:452
1267999118,2cyclictest900-21grep09:45:161
1267999118,2cyclictest5108-21df_abs11:05:161
1267999118,2cyclictest4371-21expr09:05:171
1267999118,2cyclictest31537-21ssh09:25:031
1267999118,2cyclictest28359-21cat08:50:001
1267999118,2cyclictest26049-21sort08:45:011
1267999118,2cyclictest1120-21date09:00:001
1267999114,5cyclictest16698-21tr08:25:151
1267999111,1cyclictest6118-21diskmemload10:04:231
12679991110,1cyclictest0-21swapper/110:10:121
1267299119,1cyclictest5003-21grep10:45:230
1267299119,1cyclictest32057-21tr12:00:150
1267299119,1cyclictest12141-21tr10:50:140
1267299118,2cyclictest5285-21grep08:00:170
1267299118,2cyclictest4217-21cron08:00:010
1267299118,2cyclictest3702-21ssh09:27:190
1267299118,2cyclictest30210-21cut11:20:160
1267299118,2cyclictest26879-21ssh11:18:200
1267299118,2cyclictest25569-21df07:35:160
1267299118,2cyclictest25528-21tr10:00:140
1267299118,2cyclictest23395-21grep07:30:180
1267299118,2cyclictest20565-21tr07:25:150
1267299118,2cyclictest18657-21wc08:30:000
1267299118,2cyclictest18347-21ssh12:10:300
1267299118,2cyclictest11425-21sh08:15:000
1267299117,3cyclictest15014-21cron07:15:010
1267299110,10cyclictest27518-21ssh09:41:240
1268699108,2cyclictest24223-21expr08:40:143
1268699108,1cyclictest9538-21wc08:10:153
1268699108,1cyclictest65632sleep308:03:393
1268699108,1cyclictest4268-21sendmail-msp08:00:003
1268699107,2cyclictest27859-21expr07:40:153
1268699107,2cyclictest25249-21cut07:35:153
1268699107,2cyclictest18499-21fschecks_count07:20:173
1268699100,9cyclictest26599-21expr08:45:143
1268299108,1cyclictest7268-21ls08:05:172
1268299108,1cyclictest1960-21cat09:00:152
1268299108,1cyclictest18932-21ssh10:35:012
1268299108,1cyclictest16582-21cat07:15:192
1268299107,2cyclictest9392-21grep08:10:142
1268299107,2cyclictest4494-21taskset08:00:122
1268299107,2cyclictest31599-21tr08:55:142
1268299107,2cyclictest26284-21cut08:45:132
1268299107,2cyclictest26280-21wc07:35:222
1268299105,4cyclictest23266-21fschecks_count07:30:172
1268299105,3cyclictest21171-21cron08:35:002
1267999108,1cyclictest57192sleep108:00:211
1267999108,1cyclictest2244-21cut07:55:141
1267299108,1cyclictest18510-21cut07:20:170
1267299108,1cyclictest15025-21grep08:20:180
1267299107,2cyclictest601-21cat07:50:180
1267299107,2cyclictest31158-21cat08:55:000
1267299107,2cyclictest29128-21sed07:40:270
1267299107,2cyclictest27826-21sed08:45:250
1267299107,2cyclictest22037-21uniq08:35:160
126829997,1cyclictest20349-21grep07:25:142
126829996,2cyclictest21493-21basename08:35:132
126729997,1cyclictest19690-21grep08:30:170
126729991,0cyclictest141rcu_preempt07:45:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional