You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-15 - 18:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Wed Jul 15, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1654229869,9sleep30-21swapper/307:07:243
1668029784,9sleep00-21swapper/007:09:100
1670129648,44sleep10-21swapper/107:09:271
1656128371,8sleep20-21swapper/207:07:402
283212650,0sleep00-21swapper/009:59:430
16829992726,1cyclictest0-21swapper/011:39:200
1682999270,3cyclictest0-21swapper/011:14:380
16833992524,1cyclictest0-21swapper/107:28:141
1683399250,23cyclictest0-21swapper/109:37:301
1682999240,23cyclictest0-21swapper/010:46:170
1683399230,17cyclictest0-21swapper/109:31:471
1682999230,1cyclictest0-21swapper/012:06:190
1683999220,1cyclictest0-21swapper/309:12:333
1683699220,22cyclictest0-21swapper/211:39:392
1683699220,19cyclictest0-21swapper/212:27:362
1683399226,10cyclictest24709-21runrttasks12:12:161
1682999220,20cyclictest0-21swapper/011:20:080
16833992118,1cyclictest0-21swapper/110:38:011
1683999200,20cyclictest0-21swapper/311:13:593
16836992017,2cyclictest32-21ksoftirqd/212:00:002
1683399201,6cyclictest0-21swapper/107:35:151
16829992019,1cyclictest0-21swapper/012:36:190
1682999200,0cyclictest0-21swapper/009:26:200
16829991915,1cyclictest0-21swapper/012:28:230
1683999183,4cyclictest32116-21perf11:00:013
16839991817,1cyclictest0-21swapper/309:54:573
1683999180,3cyclictest453-21ssh12:16:573
1683999180,1cyclictest0-21swapper/310:40:153
1683999180,17cyclictest0-21swapper/311:20:203
1683399182,14cyclictest15607-21ssh10:10:311
16833991816,1cyclictest0-21swapper/112:39:181
16833991815,1cyclictest0-21swapper/109:46:301
1683399181,16cyclictest28111-21cron08:40:011
1682999182,9cyclictest4134-21/usr/sbin/munin09:45:220
1682999180,18cyclictest0-21swapper/011:57:130
1683999170,11cyclictest225-21systemd-journal08:48:423
1683699173,4cyclictest1142-21runrttasks11:40:142
1683699170,1cyclictest0-21swapper/210:07:342
1683699170,17cyclictest0-21swapper/211:18:152
1683699170,17cyclictest0-21swapper/211:13:582
1683399171,1cyclictest0-21swapper/112:22:461
1683999160,16cyclictest0-21swapper/310:17:073
1683999160,15cyclictest0-21swapper/312:28:243
1683999160,0cyclictest0-21swapper/309:29:333
16836991615,1cyclictest4586-21sed12:20:012
1683699161,12cyclictest31333-21runrttasks10:00:292
1683699160,16cyclictest0-21swapper/209:57:532
16833991616,0cyclictest0-21swapper/109:20:241
1683399160,16cyclictest0-21swapper/110:56:121
16829991614,1cyclictest31565-21apt-get10:40:140
1682999160,16cyclictest0-21swapper/011:25:470
16836991513,1cyclictest31565-21apt-get10:40:142
1683699150,3cyclictest0-21swapper/212:22:402
1683699150,1cyclictest141rcu_preempt10:19:402
1683699150,1cyclictest0-21swapper/209:54:572
1683699150,13cyclictest5921-21grep10:25:012
1683399151,1cyclictest141rcu_preempt11:35:491
16833991511,2cyclictest172532sleep109:52:351
16833991510,3cyclictest23491-21grep08:30:111
1683399150,3cyclictest0-21swapper/110:20:121
1683399150,14cyclictest0-21swapper/111:42:441
1682999150,1cyclictest0-21swapper/012:00:460
16839991410,2cyclictest26543-21ls07:29:593
1683999140,6cyclictest0-21swapper/309:46:423
1683699142,11cyclictest16009-21ssh09:32:242
16836991411,2cyclictest24405-21ssh10:35:212
16836991410,3cyclictest19486-21df_abs09:15:152
16836991410,3cyclictest17640-21cat08:15:262
1683699140,1cyclictest0-21swapper/210:55:162
1683399142,4cyclictest225-21systemd-journal11:49:131
1683399141,9cyclictest26937-21apt11:55:001
1683399141,12cyclictest31836-21grep10:40:161
16833991411,2cyclictest15091-21wc12:25:271
16833991410,3cyclictest5827-21tr09:00:141
16833991410,3cyclictest31148-21ssh10:00:261
1683399140,1cyclictest0-21swapper/109:12:291
1682999147,3cyclictest5531-21/usr/sbin/munin09:00:120
1682999146,7cyclictest30222-21apache211:06:270
16829991411,3cyclictest0-21swapper/009:13:180
16829991411,2cyclictest1360-21wc11:00:180
16829991410,3cyclictest24644-21grep08:30:190
1683999138,2cyclictest18868-21grep08:20:153
1683999134,8cyclictest20036-21/usr/sbin/munin09:35:153
16839991312,1cyclictest0-21swapper/309:15:283
16839991310,2cyclictest8892-21wc10:45:213
16839991310,2cyclictest31862-21tr07:40:143
16839991310,2cyclictest22665-21grep08:25:263
16839991310,2cyclictest17726-21grep10:50:273
16839991310,2cyclictest14452-21sed08:10:183
16839991310,2cyclictest13681-21grep10:10:143
16839991310,2cyclictest12072-21ntpq11:45:213
1683999130,1cyclictest0-21swapper/310:07:113
1683699139,3cyclictest30811-21tail12:35:182
1683699131,11cyclictest0-21swapper/209:35:162
16836991310,2cyclictest8947-21grep08:00:152
16836991310,2cyclictest30175-21grep09:40:272
16836991310,2cyclictest27527-21grep07:30:182
16836991310,2cyclictest19497-21tr11:30:222
1683699130,2cyclictest0-21swapper/211:22:042
1683699130,2cyclictest0-21swapper/209:24:262
1683699130,1cyclictest0-21swapper/212:33:122
1683399139,3cyclictest28715-21sendmail-msp10:00:011
1683399139,2cyclictest21331-21tr08:25:151
1683399138,4cyclictest18859-21ssh10:51:151
1683399138,3cyclictest10550irq/122-ahci[0011:24:591
1683399131,7cyclictest0-21swapper/110:27:311
1683399131,1cyclictest141rcu_preempt12:06:421
16833991310,2cyclictest9913-21cut11:25:121
16833991310,2cyclictest27559-21cut07:30:181
16833991310,2cyclictest25-21ksoftirqd/112:00:101
16833991310,2cyclictest10044-21wc11:05:251
1683399130,13cyclictest0-21swapper/110:32:141
1683399130,12cyclictest495-21cron08:50:001
1682999139,3cyclictest2902-21sh08:55:010
16829991310,2cyclictest7219-21cut10:25:180
16829991310,2cyclictest29362-21awk12:34:590
1682999130,12cyclictest5916-21dump-pmu-power10:25:010
1683999129,2cyclictest6630-21tr10:25:153
1683999129,2cyclictest6356-21uname12:20:223
1683999129,2cyclictest4333-21grep12:00:183
1683999129,2cyclictest31022-21perf10:40:003
1683999129,2cyclictest29669-21taskset09:20:463
1683999129,2cyclictest29451-21grep09:40:203
1683999129,2cyclictest28671-21idleruntime-cro10:00:013
1683999129,2cyclictest22329-21grep12:30:153
1683999129,2cyclictest22310-21grep07:20:153
1683999129,2cyclictest10847-21basename08:05:003
1683999121,7cyclictest19161-21cron07:15:013
1683699129,2cyclictest8302-21grep10:45:182
1683699129,2cyclictest3903-21grep09:25:162
1683699129,2cyclictest3808-21grep12:00:152
1683699129,2cyclictest30906-21tail07:35:282
1683699129,2cyclictest28186-21cron12:15:002
1683699129,2cyclictest23292-21chrt11:52:092
1683699129,2cyclictest2255-21grep07:45:172
1683699129,2cyclictest21864-21date07:20:012
1683699129,2cyclictest17756-21cut07:10:172
1683699129,2cyclictest16367-21cron11:10:002
1683699129,2cyclictest14976-21grep10:30:122
1683699127,4cyclictest5207-21interrupts09:45:202
16836991210,1cyclictest0-21swapper/210:53:132
1683699120,11cyclictest12881-21grep12:05:172
1683699120,11cyclictest0-21swapper/211:03:212
1683399129,2cyclictest5524-21tr10:05:161
1683399129,2cyclictest3923-21cut09:25:161
1683399129,2cyclictest3203-21grep07:45:251
1683399129,2cyclictest29511-21grep08:40:191
1683399129,2cyclictest21805-21cat07:20:001
1683399129,2cyclictest19280-21cut08:20:161
1683399129,2cyclictest15308-21sed08:10:271
1683399128,2cyclictest25741-21tr11:15:141
1683399127,3cyclictest1857-21ls08:50:221
1683399124,7cyclictest10058-21diskmemload11:55:141
16833991211,1cyclictest38150irq/131-snd_hda09:16:551
1682999129,2cyclictest28067-21cut07:30:230
1682999129,2cyclictest27006-21sort08:35:190
1682999129,2cyclictest25752-21grep11:15:140
1682999129,2cyclictest22756-21grep07:20:170
1682999129,2cyclictest2228-21grep07:45:160
1682999129,2cyclictest18528-21grep11:30:170
1682999129,2cyclictest17685-21grep08:15:270
1682999129,2cyclictest16694-21grep10:30:250
1682999128,3cyclictest5311-21basename09:00:000
1682999128,3cyclictest29781-21mailstats09:40:230
1682999128,3cyclictest28052-21grep09:20:190
1682999126,4cyclictest2365-21/usr/sbin/munin11:40:170
68112110,1sleep30-21swapper/307:55:173
1683999118,2cyclictest6301-21head09:00:163
1683999118,2cyclictest5156-21taskset08:58:393
1683999118,2cyclictest4705-21rm11:40:403
1683999118,2cyclictest32738-21ssh11:19:323
1683999118,2cyclictest32546-21ssh12:35:543
1683999118,2cyclictest32315-21sh10:20:273
1683999118,2cyclictest3217-21sed07:45:273
1683999118,2cyclictest31723-21ssh10:00:483
1683999118,2cyclictest28938-21cat07:34:593
1683999118,2cyclictest28544-21taskset11:35:323
1683999118,2cyclictest27698-21wc11:55:153
1683999118,2cyclictest22609-21sh12:10:373
1683999118,2cyclictest20882-21cat11:50:263
1683999118,2cyclictest20378-21ssh10:33:173
1683999118,2cyclictest18480-21grep11:30:163
1683999118,2cyclictest13376-21ssh11:26:413
1683999118,2cyclictest11681-21readlink12:05:003
1683999118,2cyclictest11448-21acpi09:30:113
1683999118,2cyclictest11380-21ssh11:06:063
1683999110,10cyclictest1686-21ssh11:00:213
1683699119,1cyclictest3398-21tr08:55:152
1683699118,2cyclictest31493-21sort08:45:192
1683699118,2cyclictest23563-21tr08:30:112
1683699118,2cyclictest19897-21ls08:20:222
1683699118,2cyclictest15767-21chrt09:12:572
1683699118,2cyclictest11068-21chrt10:27:302
1683699118,2cyclictest11029-21ssh11:25:182
1683699117,3cyclictest11560-21grep11:45:172
1683699110,10cyclictest16807-21rm10:11:272
1683699110,10cyclictest14142-21expr08:10:162
1683399118,2cyclictest8258-21grep10:45:181
1683399118,2cyclictest8215-21grep09:05:171
1683399118,2cyclictest4590-21ssh11:02:191
1683399118,2cyclictest4203-21tr07:50:151
1683399118,2cyclictest32562-21tr07:40:181
1683399118,2cyclictest28050-21sh09:40:061
1683399118,2cyclictest22096-21sh11:12:551
1683399118,2cyclictest21610-21ssh10:15:021
1683399118,2cyclictest18522-21grep11:30:171
1683399118,2cyclictest16072-21chrt08:15:141
1683399118,2cyclictest11056-21ls08:05:011
1683399114,3cyclictest225-21systemd-journal12:15:011
1683399111,8cyclictest0-21swapper/112:30:111
1682999118,2cyclictest986-21ssh12:17:210
1682999118,2cyclictest8347-21expr09:05:170
1682999118,2cyclictest7113-21ssh12:20:360
1682999118,2cyclictest3951-21apache_accesses07:50:130
1682999118,2cyclictest32371-21grep07:40:160
1682999118,2cyclictest30561-21ssh10:00:200
1682999118,2cyclictest29232-21aten2_r7power_e07:35:150
1682999118,2cyclictest28545-21grep08:40:140
1682999118,2cyclictest25980-21ssh10:36:080
1682999118,2cyclictest22084-21expr08:25:200
1682999118,2cyclictest21857-21grep09:35:290
1682999118,2cyclictest21191-21ssh11:50:310
1682999118,2cyclictest19669-21wc08:20:190
1682999118,2cyclictest15664-21ssh10:50:070
1682999118,2cyclictest15542-21ssh11:47:370
1682999118,2cyclictest15359-21grep08:10:260
1682999117,2cyclictest6364-21grep07:55:150
1682999115,5cyclictest7728-21munin-node10:15:020
1682999114,6cyclictest13521-21apt-get10:10:130
1682999112,8cyclictest19166-21cron07:15:000
1682999112,6cyclictest10058-21diskmemload10:55:180
1682999111,5cyclictest0-21swapper/009:50:140
1682999110,10cyclictest21453-21ssh09:15:520
1682999110,10cyclictest112722sleep009:30:020
1683999108,1cyclictest7902-21chrt09:05:053
1683999108,1cyclictest4334-21cat07:50:153
1683999108,1cyclictest30487-21chrt08:44:013
1683999107,2cyclictest26249-21grep08:35:163
1683999107,2cyclictest25238-21chrt08:31:183
1683999107,2cyclictest21100-21sed07:15:273
1683999107,2cyclictest17903-21taskset08:16:373
1683999107,2cyclictest12068-21tail08:05:183
1683699108,1cyclictest8137-21ls09:05:172
1683699108,1cyclictest6927-21fschecks_count07:55:182
1683699108,1cyclictest59572sleep207:53:472
1683699108,1cyclictest278432sleep208:36:352
1683699108,1cyclictest13139-21taskset08:06:292
1683699107,2cyclictest7158-21cat09:00:252
1683699107,2cyclictest24505-21aten2_r7power_p07:25:132
1683699107,2cyclictest22814-21sed07:20:172
1683699100,9cyclictest32551-21uname07:40:182
1683399108,1cyclictest11651-21expr08:05:151
1683399107,2cyclictest8216-21chrt07:57:191
1683399104,5cyclictest24155-21cron07:25:001
1682999108,1cyclictest9553-21basename08:00:190
1682999108,1cyclictest5035-21ssh10:05:070
1682999108,1cyclictest264622sleep007:29:140
1682999108,1cyclictest20280-21cat07:15:190
1682999107,2cyclictest466-21taskset08:49:500
1682999107,2cyclictest21114-21grep12:10:160
1682999105,2cyclictest225-21systemd-journal08:10:000
168399997,1cyclictest3034-21cut08:55:003
168399997,1cyclictest29456-21expr07:35:163
168369997,1cyclictest21248-21wc08:25:142
168369996,2cyclictest28879-21expr08:40:142
168369996,2cyclictest1056-21expr08:50:172
168339997,1cyclictest17715-21grep07:10:161
168339996,2cyclictest3658-21df08:55:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional