You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-17 - 22:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Thu Sep 17, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1716129466,8sleep20-21swapper/207:07:432
1713829480,9sleep00-21swapper/007:07:250
1701028777,6sleep10-21swapper/107:05:451
1713628168,9sleep30-21swapper/307:07:243
70392770,2sleep339-21ksoftirqd/312:39:343
49732470,0sleep10-21swapper/111:40:121
17422993229,2cyclictest25-21ksoftirqd/109:13:501
1742899282,2cyclictest10011-21diskmemload10:50:233
1742599280,27cyclictest0-21swapper/211:15:002
317792260,2sleep231725-21ssh12:35:002
1742899260,4cyclictest0-21swapper/312:28:003
1742899255,1cyclictest1-21systemd09:13:183
1742899245,4cyclictest55850irq/129-enp2s0-09:53:553
1742899234,2cyclictest10011-21diskmemload09:44:293
1742899232,3cyclictest10011-21diskmemload12:32:253
17422992322,1cyclictest0-21swapper/109:55:011
17422992319,3cyclictest25-21ksoftirqd/109:20:121
1742899224,3cyclictest16945-21rm12:05:503
1742899221,1cyclictest0-21swapper/311:47:113
17422992213,3cyclictest25-21ksoftirqd/110:06:221
1742599210,19cyclictest0-21swapper/211:36:182
1742299210,0cyclictest0-21swapper/109:35:081
1741899210,20cyclictest0-21swapper/011:40:070
17425992018,1cyclictest16881-21pmu-power12:25:222
1742599201,18cyclictest0-21swapper/209:44:382
17422992019,1cyclictest0-21swapper/110:32:261
17422992018,2cyclictest25-21ksoftirqd/110:55:081
17422992018,1cyclictest25-21ksoftirqd/112:11:331
1742299200,2cyclictest10011-21diskmemload11:46:281
1742899191,2cyclictest0-21swapper/310:23:013
1742599192,7cyclictest0-21swapper/210:26:162
17422991914,2cyclictest25-21ksoftirqd/110:27:231
1742299190,2cyclictest0-21swapper/111:15:131
1742899183,5cyclictest10011-21diskmemload11:30:263
1742899183,2cyclictest10011-21diskmemload11:05:273
1742899183,1cyclictest13041-21taskset10:27:183
1742899181,6cyclictest9811-21grep08:00:143
1742899181,1cyclictest0-21swapper/309:05:143
1742299183,6cyclictest0-21swapper/109:30:101
17422991817,1cyclictest28123-21kworker/u8:211:01:571
1742299181,17cyclictest10011-21diskmemload11:12:111
1742299180,3cyclictest0-21swapper/112:17:511
17418991818,0cyclictest0-21swapper/009:35:070
1742899179,7cyclictest4269-21cron09:25:003
1742899179,7cyclictest24616-21head08:30:213
1742899178,8cyclictest26604-21ssh09:56:323
1742899178,8cyclictest18762-21head07:10:193
1742899178,2cyclictest1377-21chrt07:41:013
1742899174,5cyclictest225-21systemd-journal09:16:593
1742899174,5cyclictest1037-21gdbus10:55:083
1742899173,9cyclictest22693-21/usr/sbin/munin12:10:203
1742899172,3cyclictest10011-21diskmemload10:45:213
1742899170,16cyclictest20793-21sed07:15:173
1742299172,14cyclictest25306-21irqstats10:15:191
17422991716,1cyclictest0-21swapper/111:30:011
17422991715,2cyclictest25-21ksoftirqd/110:21:251
17422991715,1cyclictest10748-21grep08:00:241
1742299170,1cyclictest0-21swapper/111:53:141
17418991717,0cyclictest0-21swapper/009:15:430
1742899169,6cyclictest31291-21sed08:45:163
1742899168,7cyclictest2582-21cut07:45:163
1742899168,7cyclictest17770-21grep10:30:163
1742899168,2cyclictest3997-21ssh10:02:393
1742899168,2cyclictest22707-21tr07:20:143
1742899168,2cyclictest21111-21ssh09:34:583
1742899168,2cyclictest11258-21ssh10:06:473
1742899167,2cyclictest3811-21ssh11:39:373
1742899167,2cyclictest2598-21ssh10:40:193
1742899162,4cyclictest225-21systemd-journal11:25:003
1742899161,1cyclictest0-21swapper/308:40:143
1742899160,9cyclictest5897-21taskset12:00:063
1742599163,8cyclictest0-21swapper/211:56:072
17425991611,3cyclictest6110-21ssh12:00:132
1742599160,16cyclictest0-21swapper/210:18:342
17422991613,2cyclictest25-21ksoftirqd/109:47:431
1742299160,1cyclictest25-21ksoftirqd/108:20:271
1742299160,1cyclictest0-21swapper/112:06:031
17418991615,1cyclictest0-21swapper/011:15:120
1742899159,5cyclictest6091-21grep11:40:193
1742899159,5cyclictest3493-21cut11:00:153
1742899158,2cyclictest21781-21ssh10:13:273
1742899158,2cyclictest21781-21ssh10:13:273
1742899157,2cyclictest8021-21ssh09:26:133
1742899157,2cyclictest27471-21cat11:15:003
1742899157,2cyclictest13736-21grep11:25:213
1742899157,1cyclictest27385-21taskset09:38:103
1742899154,4cyclictest7778-21mailstats09:45:233
1742899152,4cyclictest225-21systemd-journal08:37:573
1742899152,2cyclictest225-21systemd-journal07:40:003
1742899152,1cyclictest0-21swapper/312:25:003
1742899151,2cyclictest225-21systemd-journal08:30:003
1742899151,2cyclictest225-21systemd-journal08:30:003
1742599150,2cyclictest0-21swapper/211:21:572
1742599150,11cyclictest11078-21/usr/sbin/munin11:05:202
1742599150,0cyclictest0-21swapper/209:34:512
1742299150,2cyclictest0-21swapper/110:37:261
1742299150,14cyclictest21710-21ssh10:51:401
1741899153,6cyclictest29310-21idleruntime-cro07:35:000
17418991513,1cyclictest17409-21cyclictest08:15:130
17418991511,3cyclictest27857-21cron08:40:000
1742899148,1cyclictest1152-21ssh12:15:453
1742899147,2cyclictest27635-21ssh10:16:313
1742899147,2cyclictest25480-21ssh11:51:493
1742899147,1cyclictest30626-21sh11:55:133
1742899147,1cyclictest2773-21chrt11:19:363
1742899146,2cyclictest3882-21grep08:55:163
1742899146,2cyclictest29211-21chrt07:34:083
1742899141,6cyclictest225-21systemd-journal10:40:013
1742899141,4cyclictest225-21systemd-journal08:00:003
17425991411,2cyclictest20204-21wc08:20:282
17425991410,3cyclictest2454-21grep07:45:152
1742599140,6cyclictest0-21swapper/210:45:132
1742599140,1cyclictest0-21swapper/207:40:112
17422991413,1cyclictest25-21ksoftirqd/108:10:161
1742299141,2cyclictest9614-21ssh10:45:071
1742299141,1cyclictest141rcu_preempt11:22:401
17422991411,3cyclictest25-21ksoftirqd/110:02:271
1741899146,6cyclictest31570-21apache209:54:340
1741899144,9cyclictest10022-21perf09:09:590
17418991410,2cyclictest30196-21grep09:20:210
17418991410,2cyclictest25505-21apt08:35:000
1741899140,3cyclictest0-21swapper/011:37:340
1741899140,1cyclictest0-21swapper/009:13:150
1741899140,14cyclictest0-21swapper/012:14:210
1742899139,3cyclictest25797-21grep07:25:193
1742899137,1cyclictest1837-21uname08:50:233
1742899137,1cyclictest114372sleep308:05:013
1742899136,2cyclictest17540-21awk08:15:223
1742899136,1cyclictest55072sleep309:00:113
1742599139,3cyclictest22941-21wc11:50:212
1742599139,3cyclictest12236-21sed08:05:162
1742599136,6cyclictest15239-21grep12:05:172
1742599131,11cyclictest0-21swapper/210:10:142
1742599131,11cyclictest0-21swapper/210:10:132
17425991310,3cyclictest32-21ksoftirqd/211:15:212
17425991310,2cyclictest7416-21grep07:55:162
17425991310,2cyclictest5916-21cut11:40:172
17425991310,2cyclictest5138-21ssh09:25:132
1742599130,1cyclictest0-21swapper/211:01:172
1742599130,1cyclictest0-21swapper/210:43:262
1742599130,13cyclictest0-21swapper/211:34:132
1742599130,11cyclictest0-21swapper/209:22:402
1742599130,10cyclictest0-21swapper/210:57:112
1742299139,3cyclictest29123-21cut11:35:131
1742299138,5cyclictest25-21ksoftirqd/107:40:111
1742299138,4cyclictest16491-21sh10:10:161
1742299138,4cyclictest16491-21sh10:10:161
1742299134,6cyclictest25-21ksoftirqd/109:43:551
17422991312,1cyclictest241rcuc/108:40:171
17422991310,2cyclictest32122-21cat08:45:251
17422991310,2cyclictest28269-21cron09:20:001
1742299130,12cyclictest20945-21taskset12:28:101
1741899139,3cyclictest2987-21cut11:00:130
1741899139,3cyclictest12990-21grep11:25:160
1741899134,3cyclictest141rcu_preempt12:33:230
17418991310,2cyclictest7030-21grep09:45:160
17418991310,2cyclictest5552-21grep07:50:200
17418991310,2cyclictest3921-21ssh11:20:140
1741899130,2cyclictest0-21swapper/010:20:260
1741899130,12cyclictest0-21swapper/010:28:290
1742899129,2cyclictest14617-21cut08:10:173
1742899126,1cyclictest5729-21chrt07:50:233
1742899126,1cyclictest18592-21chrt08:20:043
1742599129,2cyclictest9023-21sed12:20:252
1742599129,2cyclictest8797-21grep10:05:202
1742599129,2cyclictest779-21tr08:50:142
1742599129,2cyclictest6609-21cron12:20:002
1742599129,2cyclictest2597-21chrt12:36:042
1742599129,2cyclictest24807-21sh09:55:252
1742599129,2cyclictest23789-21grep09:35:262
1742599129,2cyclictest23109-21grep07:20:172
1742599129,2cyclictest18393-21grep07:10:162
1742599129,2cyclictest12492-21tr11:25:132
1742599129,2cyclictest10000-21sh09:10:002
1742599129,2cyclictest0-21swapper/209:13:182
1742599128,3cyclictest24482-21grep08:30:192
1742599128,3cyclictest21287-21grep08:25:132
1742599128,3cyclictest21287-21grep08:25:132
1742599128,3cyclictest17394-21ssh10:30:152
1742599128,1cyclictest25791-21expr10:35:182
17425991210,1cyclictest0-21swapper/207:25:152
1742299129,2cyclictest9549-21sed09:05:271
1742299129,2cyclictest7717-21cat12:00:251
1742299129,2cyclictest28725-21cat07:30:251
1742299129,2cyclictest26045-21head07:25:211
1742299129,2cyclictest25762-21grep12:30:261
1742299129,2cyclictest1630-21cat12:35:261
1742299129,2cyclictest15045-21ssh09:50:141
1742299127,5cyclictest38150irq/131-snd_hda07:10:121
17422991211,1cyclictest0-21swapper/109:28:541
1742299121,10cyclictest10011-21diskmemload11:31:121
17422991210,2cyclictest25-21ksoftirqd/110:40:191
1741899129,2cyclictest30199-21ssh11:55:110
1741899129,2cyclictest29792-21tr07:35:140
1741899129,2cyclictest22149-21sh07:20:000
1741899129,2cyclictest21743-21grep08:25:160
1741899129,2cyclictest21743-21grep08:25:150
1741899129,2cyclictest18443-21head07:10:160
1741899129,2cyclictest17280-21tr10:30:150
1741899126,2cyclictest13-21ksoftirqd/010:45:060
1741899125,6cyclictest10011-21diskmemload09:57:290
1741899120,8cyclictest0-21swapper/011:50:140
1741899120,11cyclictest0-21swapper/009:32:580
1742599118,2cyclictest8896-21ssh09:45:562
1742599118,2cyclictest31648-21grep10:00:132
1742599118,2cyclictest31013-21grep08:45:142
1742599118,2cyclictest28504-21wc07:30:222
1742599118,2cyclictest21131-21sort09:15:162
1742599118,2cyclictest18061-21ssh09:51:252
1742599118,2cyclictest1751-21sh10:20:212
1742599118,2cyclictest15907-21ssh11:46:002
1742599116,2cyclictest18153-21apache_volume10:50:142
1742299118,2cyclictest432-21ssh11:56:021
1742299118,2cyclictest3668-21sed07:45:261
1742299118,2cyclictest2638-21sh08:55:011
1742299118,2cyclictest23220-21head07:20:171
1742299118,2cyclictest22714-21grep08:25:261
1742299118,2cyclictest22714-21grep08:25:261
1742299118,2cyclictest22123-21chrt07:19:511
1742299118,2cyclictest13070-21grep11:05:281
1742299111,8cyclictest13607-21idleruntime-cro08:10:001
1741899119,1cyclictest7409-21grep12:20:140
1741899119,1cyclictest12404-21ssh11:05:220
1741899118,2cyclictest8946-21sh12:01:060
1741899118,2cyclictest8869-21ssh10:05:210
1741899118,2cyclictest806-21sh10:40:010
1741899118,2cyclictest5771-21head09:25:160
1741899118,2cyclictest5743-21basename09:45:000
1741899118,2cyclictest3622-21sed07:45:260
1741899118,2cyclictest30953-21chrt12:15:070
1741899118,2cyclictest28390-21grep08:40:150
1741899118,2cyclictest26358-21systemd10:55:070
1741899118,2cyclictest25587-21ssh10:15:220
1741899118,2cyclictest25111-21ssh10:54:190
1741899118,2cyclictest24070-21grep07:20:270
1741899118,2cyclictest2357-21uname10:40:180
1741899118,2cyclictest23151-21ssh11:11:440
1741899118,2cyclictest2077-21ssh12:35:390
1741899118,2cyclictest20378-21ssh11:30:070
1741899118,2cyclictest19836-21seq10:11:560
1741899118,2cyclictest19836-21seq10:11:560
1741899118,2cyclictest1891-21ssh10:00:590
1741899118,2cyclictest15241-21tail12:05:170
1741899118,2cyclictest12934-21ssh11:45:030
1741899117,3cyclictest11597-21basename08:05:130
1742599108,1cyclictest5310-21sed07:50:172
1742599108,1cyclictest29648-21aten2_r7power_p07:35:132
1742599107,2cyclictest6843-21awk09:00:222
1742599107,2cyclictest28747-21fschecks_count08:40:172
1742599107,2cyclictest26302-21sed08:35:162
1742599107,2cyclictest22575-21ssh12:10:062
1742599107,2cyclictest20272-21grep07:15:142
1742599107,2cyclictest10766-21grep08:00:242
1742299108,2cyclictest6093-21dnf09:00:151
1742299108,1cyclictest7542-21basename07:55:171
1742299108,1cyclictest4955-21taskset08:56:381
1742299108,1cyclictest4302-21taskset07:50:081
1742299108,1cyclictest30563-21sed07:35:191
1742299107,2cyclictest9214-21rm12:20:291
1742299107,2cyclictest26887-21grep08:35:221
1742299107,2cyclictest23605-21grep08:30:131
1741899108,1cyclictest30733-21cat08:45:140
1741899108,1cyclictest185992sleep008:20:090
1741899107,2cyclictest7430-21tail07:55:160
1741899107,2cyclictest5650-21grep09:00:130
1741899107,2cyclictest15325-21ssh12:25:070
1741899107,2cyclictest10487-21sensors_fan08:00:210
1741899106,3cyclictest24880-21tr07:25:130
1741899104,5cyclictest25478-21kworker/0:107:40:120
1741899102,5cyclictest0-21swapper/008:10:140
1741899100,9cyclictest3458-21cat08:55:140
174259997,1cyclictest3256-21wget08:55:132
174259996,2cyclictest13923-21grep08:10:142
174259992,4cyclictest17555-21sendmail_mailqu08:15:222
174229996,2cyclictest16440-21cat08:15:141
174189998,1cyclictest1697-21ntpq08:50:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional