You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-19 - 06:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Fri Dec 19, 2025 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1058028658,9sleep30-21swapper/319:09:503
1033028371,8sleep20-21swapper/219:06:352
1040728269,9sleep00-21swapper/019:07:330
1049628059,6sleep10-21swapper/119:08:431
10688992722,4cyclictest32-21ksoftirqd/219:50:002
10688992722,4cyclictest32-21ksoftirqd/219:50:002
1068399248,6cyclictest225-21systemd-journal23:00:000
1068899231,18cyclictest0-21swapper/222:15:222
1068599230,22cyclictest0-21swapper/122:40:141
1068399173,8cyclictest29754-21munin-run19:50:010
1068399173,8cyclictest29754-21munin-run19:50:000
1068899163,1cyclictest0-21swapper/223:00:142
1068899161,1cyclictest0-21swapper/221:30:152
10685991611,2cyclictest25-21ksoftirqd/121:55:161
10690991510,1cyclictest12979-21cron23:45:003
1068899159,4cyclictest32-21ksoftirqd/223:45:132
1068899154,2cyclictest141rcu_preempt21:15:272
1068899152,1cyclictest141rcu_preempt22:10:132
10688991512,2cyclictest32-21ksoftirqd/222:05:002
10688991510,2cyclictest32-21ksoftirqd/219:20:202
1068899150,14cyclictest21360-21grep19:30:172
10685991511,3cyclictest23453-21cut21:50:171
1068599150,2cyclictest0-21swapper/119:10:131
1068399153,10cyclictest6842-21idleruntime-cro20:10:000
1068399152,7cyclictest16601-21idleruntime-cro22:45:000
1069099146,7cyclictest0-21swapper/322:40:143
1068899149,4cyclictest8229-21perf23:35:002
1068899149,4cyclictest22756-21apache222:25:182
1068899145,8cyclictest24977-21apt19:40:012
1068899142,7cyclictest32-21ksoftirqd/223:20:232
10688991411,2cyclictest5900-21wc22:20:192
10688991410,3cyclictest3556-21grep21:10:152
10688991410,3cyclictest25952-21grep21:55:172
10688991410,2cyclictest2613-21grep00:30:142
1068399144,5cyclictest8235-21perf23:35:000
1068399141,7cyclictest20127-21munin-run00:00:010
1068399141,7cyclictest10629-21grep23:40:000
1069099137,3cyclictest11894-21apt-get20:20:173
1069099134,7cyclictest22753-21apache219:55:143
1069099134,2cyclictest225-21systemd-journal19:22:283
10690991310,2cyclictest20439-21grep20:35:243
10690991310,2cyclictest18838-21head19:25:183
1068899132,3cyclictest141rcu_preempt00:20:182
10688991310,2cyclictest6449-21grep20:05:272
10688991310,2cyclictest2587-21wc21:05:272
10688991310,2cyclictest25668-21grep20:45:262
10688991310,2cyclictest18856-21dpkg20:35:002
10688991310,2cyclictest16693-21ls23:50:212
10688991310,2cyclictest15008-21cut20:25:172
10688991310,2cyclictest10653-21wc22:30:202
1068899130,3cyclictest141rcu_preempt20:50:172
1068599138,2cyclictest4675-21cron00:35:001
1068399131,6cyclictest24996-21munin-run19:40:000
10683991310,2cyclictest712-21grep19:55:180
10683991310,2cyclictest29973-21apt00:19:590
10683991310,2cyclictest27640-21grep20:50:240
10683991310,2cyclictest26222-21cat19:40:190
10683991310,2cyclictest26145-21cron23:05:010
10683991310,2cyclictest11282-21grep22:30:280
1068399130,12cyclictest26232-21cat21:55:200
1069099129,2cyclictest6368-21grep20:05:263
1069099129,2cyclictest5099-21wc00:35:143
1069099129,2cyclictest4674-21cron00:35:003
1069099129,2cyclictest32069-21cron22:10:003
1069099129,2cyclictest18458-21grep22:45:273
1069099129,2cyclictest1741-21grep21:05:183
1069099129,2cyclictest16402-21grep23:50:183
1069099129,2cyclictest14955-21grep19:15:253
1069099129,2cyclictest10416-21basename21:25:003
1069099122,5cyclictest0-21swapper/300:00:163
10690991210,1cyclictest7993-21cron21:20:013
10690991210,1cyclictest32409-21grep21:00:243
10690991210,1cyclictest14488-21grep23:45:213
1069099120,6cyclictest0-21swapper/321:40:153
1069099120,6cyclictest0-21swapper/321:40:153
1069099120,2cyclictest0-21swapper/323:30:123
1068899129,2cyclictest853-21cron21:05:002
1068899129,2cyclictest32-21ksoftirqd/220:15:172
1068899129,2cyclictest32-21ksoftirqd/219:10:202
1068899129,2cyclictest28833-21grep23:10:152
1068899129,2cyclictest22686-21tail22:55:212
1068899129,2cyclictest22051-21grep21:45:272
1068899129,2cyclictest18296-21tr19:25:152
1068899128,3cyclictest22499-21cron00:05:002
1068899128,3cyclictest12807-21tail22:35:182
1068899125,4cyclictest311rcuc/222:50:182
1068899125,2cyclictest10675-21cyclictest23:16:312
1068899120,11cyclictest19477-21grep20:35:172
1068599129,2cyclictest9416-21tr21:20:211
1068599129,2cyclictest4242-21tail21:10:181
1068599129,2cyclictest30055-21wc23:10:231
1068599129,2cyclictest29033-21cut20:55:151
1068599129,2cyclictest19482-21tr22:50:151
1068599129,2cyclictest18949-21cron22:50:011
1068599129,2cyclictest17339-21grep19:20:241
1068599129,2cyclictest11173-21grep23:40:161
1068599128,3cyclictest4450-21chrt20:04:561
1068599128,3cyclictest27616-21sed20:50:251
1068599127,3cyclictest22061-21tail00:00:311
1068599122,8cyclictest879-21perf21:05:001
1068599121,10cyclictest1420-21sed00:25:211
10685991210,1cyclictest5832-21cron23:30:001
10685991210,1cyclictest14705-21grep21:30:291
1068599120,10cyclictest0-21swapper/119:45:141
1068599120,10cyclictest0-21swapper/119:45:131
1068399129,2cyclictest7884-21cut20:10:180
1068399129,2cyclictest32416-21cut22:10:140
1068399129,2cyclictest30778-21grep19:50:190
1068399129,2cyclictest30427-21grep20:55:280
1068399129,2cyclictest27668-21tr22:00:130
1068399129,2cyclictest21348-21awk20:39:590
1068399129,2cyclictest17592-21cat20:30:210
1068399129,2cyclictest15705-21grep21:35:160
1068399129,2cyclictest11680-21head19:10:180
1068399128,3cyclictest3754-21grep21:10:160
1068399121,2cyclictest141rcu_preempt20:40:160
1069099119,1cyclictest28622-21grep00:15:173
1069099119,1cyclictest11402-21head21:25:183
1069099118,2cyclictest9763-21wget22:30:163
1069099118,2cyclictest29304-21grep19:45:263
1069099118,2cyclictest29304-21grep19:45:263
1069099118,2cyclictest25587-21tr00:10:163
1069099118,2cyclictest24105-21grep19:35:223
1069099118,2cyclictest22931-21wc20:40:233
1069099118,2cyclictest11625-21tr20:20:003
1069099118,2cyclictest10607-21systemctl23:40:003
1069099117,3cyclictest29420-21grep23:10:183
1069099114,3cyclictest225-21systemd-journal22:15:003
1068899118,2cyclictest7826-21grep20:10:182
1068899118,2cyclictest7293-21sed00:40:002
1068899118,2cyclictest4830-21grep23:25:212
1068899118,2cyclictest32-21ksoftirqd/221:40:002
1068899118,2cyclictest32097-21awk22:10:012
1068899118,2cyclictest3034-21sort20:00:172
1068899118,2cyclictest28134-21head23:05:282
1068899118,2cyclictest24343-21wc21:50:262
1068899118,2cyclictest14383-21tail19:15:192
1068899118,2cyclictest12286-21grep21:25:272
1068899118,2cyclictest11993-21cut20:20:172
1068899118,2cyclictest11977-21tr23:40:212
1068899117,3cyclictest30291-21grep19:50:162
1068899117,3cyclictest1196-21tr19:55:232
1068899116,4cyclictest32-21ksoftirqd/222:40:132
1068899110,8cyclictest19930-21munin-run21:45:002
1068899110,8cyclictest19930-21munin-run21:45:002
1068599119,1cyclictest6727-21cut23:30:171
1068599119,1cyclictest18921-21head21:40:221
1068599119,1cyclictest18921-21head21:40:211
1068599119,1cyclictest10441-21head22:30:191
1068599118,2cyclictest8502-21sed22:25:221
1068599118,2cyclictest8108-21sed20:10:211
1068599118,2cyclictest6645-21head21:15:171
1068599118,2cyclictest5855-21cat00:35:191
1068599118,2cyclictest3290-21sort23:25:001
1068599118,2cyclictest31165-21chrt23:15:121
1068599118,2cyclictest27187-21chrt19:42:481
1068599118,2cyclictest26562-21cut23:05:151
1068599118,2cyclictest26181-21taskset00:10:181
1068599118,2cyclictest26099-21seq20:49:301
1068599118,2cyclictest23228-21cat19:35:161
1068599118,2cyclictest20448-21cut19:30:121
1068599118,2cyclictest20261-21cat21:45:011
1068599118,2cyclictest19965-21chrt19:27:291
1068599118,2cyclictest16359-21cron20:29:591
1068599118,2cyclictest16048-21sort21:35:181
1068599118,2cyclictest12323-21tr22:35:161
1068599118,2cyclictest10591-21uname20:15:211
1068599110,10cyclictest13987-21head23:45:161
1068399119,1cyclictest21500-21grep21:45:200
1068399118,2cyclictest31075-21ls00:20:190
1068399118,2cyclictest3005-21grep22:15:170
1068399118,2cyclictest29378-21dnf23:10:180
1068399118,2cyclictest2821-21expr20:00:150
1068399118,2cyclictest24525-21sed21:50:270
1068399118,2cyclictest21167-21grep19:30:150
1068399118,2cyclictest19440-21grep21:40:280
1068399118,2cyclictest19440-21grep21:40:270
1068399118,2cyclictest17387-21sed19:20:240
1068399118,2cyclictest15472-21cat23:50:000
1068399118,2cyclictest14018-21grep19:15:160
1068399118,2cyclictest11187-21sed20:15:270
1068399118,2cyclictest10038-21taskset21:20:310
1068399117,3cyclictest3329-21grep23:25:000
1068399110,10cyclictest871-21awk23:20:010
1069099108,2cyclictest946-21cut23:20:003
1069099108,1cyclictest30357-21tr00:20:153
1069099108,1cyclictest30280-21tr19:50:163
1069099108,1cyclictest2909-21taskset23:20:423
1069099108,1cyclictest27853-21expr22:00:143
1069099108,1cyclictest2506-21cut00:30:003
1069099108,1cyclictest24393-21expr23:00:153
1069099108,1cyclictest24043-21apache_processe20:45:123
1069099108,1cyclictest23603-21sed21:50:183
1069099108,1cyclictest21768-21run-parts19:30:213
1069099108,1cyclictest16949-21seq20:30:153
1069099108,1cyclictest14308-21cut20:25:143
1069099108,1cyclictest10868-21chrt19:10:013
1069099107,2cyclictest8905-21grep22:25:273
1069099107,2cyclictest29894-21strings20:55:213
1069099107,2cyclictest26426-21apt-get20:50:153
1069099107,2cyclictest23370-21grep22:55:283
1069099107,2cyclictest2322-21aten2_r7power_c22:15:133
1069099107,2cyclictest14244-21ls21:30:233
1069099105,2cyclictest225-21systemd-journal19:45:003
1069099102,5cyclictest14187-21cron22:39:593
1069099100,9cyclictest26401-21acpi23:05:123
1069099100,9cyclictest18356-21expr23:55:153
1068899108,1cyclictest26298-21sort00:10:192
1068899108,1cyclictest22002-21expr20:40:162
1068899108,1cyclictest20083-21taskset23:59:502
1068899108,1cyclictest18455-21grep22:45:282
1068899107,2cyclictest9514-21head21:20:232
1068899107,2cyclictest8774-21tr23:35:162
1068899107,2cyclictest28076-21tr00:15:152
1068899107,2cyclictest26803-21grep19:40:262
1068899107,2cyclictest25026-31su00:09:562
1068899107,2cyclictest2179-21seq00:28:272
1068599108,2cyclictest5557-21sort20:05:181
1068599108,1cyclictest8622-21cut23:35:161
1068599108,1cyclictest30031-21basename22:05:161
1068599108,1cyclictest2948-21cut22:15:161
1068599108,1cyclictest27837-21bc00:15:121
1068599108,1cyclictest20549-21awk20:35:241
1068599108,1cyclictest16325-21grep23:50:171
1068599108,1cyclictest118532sleep120:20:101
1068599108,1cyclictest10782-21grep21:25:151
1068599107,3cyclictest30264-21cut19:50:161
1068599107,2cyclictest6255-21tr22:20:241
1068599107,2cyclictest25650-21taskset23:00:281
1068599107,2cyclictest24359-21cat00:05:301
1068599107,2cyclictest21820-21cat20:40:161
1068599107,2cyclictest19448-21grep23:55:251
1068599107,2cyclictest17384-21tail20:30:191
1068599107,2cyclictest14525-21tr19:15:211
1068599107,2cyclictest1050-21ntpq22:10:211
1068599105,2cyclictest225-21systemd-journal21:06:541
1068599105,2cyclictest0-21swapper/119:55:131
1068599100,9cyclictest22565-21sed22:55:201
1068599100,7cyclictest225-21systemd-journal22:04:101
1068399108,2cyclictest6216-21expr21:15:150
1068399108,1cyclictest5506-21forks22:20:180
1068399108,1cyclictest5199-21cut00:35:160
1068399108,1cyclictest32765-21egrep00:25:140
1068399108,1cyclictest31897-21grep21:00:170
1068399108,1cyclictest2918-21expr00:30:150
1068399108,1cyclictest25757-21grep00:10:160
1068399108,1cyclictest19300-21aten2_r7power_p22:50:140
1068399108,1cyclictest17476-21taskset23:51:470
1068399108,1cyclictest11225-21expr23:40:150
1068399107,2cyclictest4497-21sed23:25:180
1068399107,2cyclictest30703-21grep22:05:180
1068399107,2cyclictest24448-21grep00:05:300
1068399107,2cyclictest24052-21aten2_r7power_c20:45:130
1068399107,2cyclictest20610-21cat00:00:170
1068399107,2cyclictest17628-21head22:45:180
1068399107,2cyclictest12639-21tail20:20:200
106909998,1cyclictest24096-21grep00:05:273
106909998,1cyclictest19195-21cat22:50:123
106909997,2cyclictest16218-21sh21:35:193
106909997,1cyclictest7074-21taskset20:10:113
106909997,1cyclictest4517-21grep23:25:193
106909997,1cyclictest3536-21hostname21:10:153
106909997,1cyclictest26299-21cat21:55:193
106909997,1cyclictest2571-21ls20:00:143
106909997,1cyclictest20406-21apt-get21:45:143
106889997,1cyclictest32-21ksoftirqd/220:55:242
106859997,1cyclictest30860-21grep00:20:191
106839998,1cyclictest8444-21ntpq22:25:210
106839998,1cyclictest281682sleep023:05:290
106839998,1cyclictest19410-21basename19:25:230
106839998,1cyclictest14675-21cat20:25:160
106839997,1cyclictest13976-21uname21:30:200
106839997,1cyclictest12158-21aten2_r7power_p22:35:160
106839996,2cyclictest1381-21grep21:05:170
106909985,2cyclictest5147-21grep22:20:153
106839987,1cyclictest11078-21expr21:25:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional