You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-13 - 10:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Thu Aug 13, 2026 00:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
258552172118,46sleep30-21swapper/319:08:503
25897217096,25sleep00-21swapper/019:09:230
258312143107,25sleep10-21swapper/119:08:321
257402138103,24sleep20-21swapper/219:07:192
99862600,6sleep2141rcu_preempt21:00:152
278332570,2sleep30-21swapper/321:20:143
121662570,3sleep00-21swapper/021:32:110
137492550,1sleep00-21swapper/023:56:210
149652540,3sleep126219-21cyclictest21:58:211
284552530,1sleep30-21swapper/322:31:533
267302530,2sleep20-21swapper/222:30:242
66182520,1sleep30-21swapper/321:28:243
308922520,2sleep10-21swapper/122:10:111
164452490,2sleep00-21swapper/021:35:190
97522480,2sleep20-21swapper/222:42:042
95872460,1sleep30-21swapper/321:00:123
231232460,1sleep30-21swapper/321:40:173
197832460,1sleep30-21swapper/322:01:233
159812460,1sleep00-21swapper/023:34:580
321462440,2sleep20-21swapper/219:20:232
250702430,1sleep00-21swapper/023:40:330
8752420,1sleep20-21swapper/222:11:222
220392420,2chrt0-21swapper/221:15:242
13482420,1sleep00-21swapper/000:11:030
313652400,2sleep00-21swapper/022:58:110
81712340,2sleep20-21swapper/219:40:222
81712340,2sleep20-21swapper/219:40:212
182852320,5sleep018284-21awk22:00:240
276122310,1sleep30-21swapper/300:30:243
195382310,2sleep00-21swapper/022:25:200
26224992618,6cyclictest30285-21/usr/sbin/munin21:45:203
151932260,3sleep10-21swapper/121:35:011
26912250,2sleep20-21swapper/223:00:222
26224992522,2cyclictest0-21swapper/319:13:383
26222992522,2cyclictest0-21swapper/119:12:451
204982250,2chrt0-21swapper/300:01:183
26224992416,5cyclictest804-21sh23:00:003
26223992418,4cyclictest28-21ksoftirqd/220:00:012
2622199240,1cyclictest0-21swapper/019:14:020
206552240,2sleep30-21swapper/323:37:553
26224992313,6cyclictest16675-21awk23:10:243
26223992315,6cyclictest29066-21sh00:31:292
2622199231,6cyclictest12650-21cat22:20:200
230302230,2sleep10-21swapper/100:03:491
181462230,4sleep30-21swapper/321:36:233
61682220,9sleep122-21ksoftirqd/119:35:221
34872220,1sleep30-21swapper/323:00:433
306322220,3sleep012-21ksoftirqd/021:45:230
289692220,2sleep10-21swapper/122:55:471
26224992214,6cyclictest26095-21ssh23:41:343
2622499221,18cyclictest12088-21sh00:20:003
2622399222,14cyclictest16425-21ssh23:35:112
26223992220,1cyclictest0-21swapper/219:13:272
26223992214,6cyclictest10082-21ls21:30:242
26223992212,7cyclictest560-21dbus-daemon22:20:242
2622399221,19cyclictest29207-21unixbench_singl21:20:252
2622399221,17cyclictest32004-21ssh22:35:112
2622299224,4cyclictest13484-21diskmemload23:42:521
26222992216,4cyclictest22-21ksoftirqd/121:19:591
26222992213,7cyclictest4515-21cron19:35:001
26221992213,7cyclictest32438-21users20:35:270
2622199220,8cyclictest13480-21awk21:10:000
20412220,2chrt0-21swapper/322:12:313
139812220,2sleep20-21swapper/223:33:002
2622499212,12cyclictest0-21swapper/320:15:123
26224992113,6cyclictest7245-21gdbus20:55:023
26224992112,6cyclictest10182-21fw_packets23:30:163
26224992111,7cyclictest21228-21users00:25:263
2622499211,13cyclictest15723-21chrt20:00:133
2622399218,11cyclictest28777-21cstates23:20:142
2622399212,13cyclictest4954-21chrt20:50:122
26223992112,7cyclictest20180-21awk22:25:252
2622399210,5cyclictest0-21swapper/223:51:072
2622299217,3cyclictest313292ssh21:22:301
2622299214,4cyclictest13484-21diskmemload00:30:091
2622299211,9cyclictest141rcu_preempt21:38:481
26222992114,4cyclictest6369-21cut00:15:171
26222992111,7cyclictest19855-21switchtime22:25:231
26221992113,6cyclictest23191-21date23:15:240
2622199211,18cyclictest5853-21ssh21:27:390
103972210,2sleep30-21swapper/321:55:143
2622499209,7cyclictest31986-21apt-get22:35:113
26224992013,4cyclictest6275-21sh21:51:243
26224992012,6cyclictest29233-21perl20:30:193
26224992012,6cyclictest14375-21cut21:10:163
26224992011,7cyclictest12744-21sensors_temp21:05:223
26224992011,6cyclictest5865-21ntp_states20:50:193
26224992010,7cyclictest9313-21seq22:18:023
26224992010,6cyclictest22916-21awk23:15:223
2622399207,11cyclictest28-21ksoftirqd/223:50:002
2622399202,13cyclictest19996-21taskset20:10:102
26223992014,4cyclictest27257-21ssh22:55:142
26223992013,5cyclictest24022-21latency_hist20:20:002
2622299209,8cyclictest560-21dbus-daemon23:35:001
2622299204,3cyclictest13484-21diskmemload22:23:151
2622299204,3cyclictest13484-21diskmemload00:24:181
2622299202,6cyclictest13203-21apt-get22:45:031
26222992013,5cyclictest52750irq/139-enp3s0-00:05:351
26222992012,6cyclictest713-21awk00:35:131
26222992012,6cyclictest5793-21memory22:15:161
2622299201,16cyclictest2635-21ssh21:25:191
26222992011,6cyclictest12186-21grep19:50:181
26222992011,5cyclictest26934-21grep22:30:251
2622199209,7cyclictest52950irq/141-enp3s0-22:05:120
2622199209,7cyclictest52950irq/141-enp3s0-22:05:110
2622199202,15cyclictest5772-21apt-get00:15:120
2622199201,6cyclictest27749-21rm21:43:580
26221992014,4cyclictest3327-21sh23:25:150
26221992012,6cyclictest31703-21pluginstate22:10:200
26221992012,6cyclictest31239-21fw_packets19:20:130
26221992012,6cyclictest28083-21ssh00:30:300
26221992012,6cyclictest25860-21seq21:18:510
26221992012,6cyclictest10030-21seq23:05:290
26221992011,6cyclictest5330-21cat19:35:140
2622199201,15cyclictest52950irq/141-enp3s0-22:15:100
26221992010,7cyclictest24656-21cut20:20:130
205432200,2sleep10-21swapper/123:37:481
287142190,4sleep10-21swapper/119:15:131
2622499199,7cyclictest31038-21ntpq23:45:193
2622499192,12cyclictest11110-21ssh23:55:133
2622499191,5cyclictest9991-21python321:30:233
26224991913,4cyclictest23051-21sh21:16:033
26224991913,4cyclictest1433-21sh23:23:573
26224991912,5cyclictest19298-21awk20:05:233
26224991912,4cyclictest7590-21sh23:28:233
26224991912,4cyclictest22751-21sh22:27:573
26224991912,4cyclictest2168-21sh00:11:493
26224991912,4cyclictest14286-21sh22:21:123
26224991912,4cyclictest10710-21sh23:06:103
26224991911,6cyclictest5043-21cut23:50:163
26224991911,6cyclictest13238-21cut00:20:163
26224991911,5cyclictest7337-21cat19:40:133
26224991911,5cyclictest7337-21cat19:40:133
26224991911,5cyclictest13484-21diskmemload22:48:103
2622399191,9cyclictest5994-21ntp_states22:15:182
2622399191,8cyclictest6519-21irqstats00:15:182
26223991914,4cyclictest28-21ksoftirqd/221:40:012
26223991913,4cyclictest636-21runrttasks23:05:022
26223991913,4cyclictest27441-21sh21:43:402
26223991913,4cyclictest11168-21sh19:49:592
26223991912,5cyclictest3924-21tr20:45:202
26223991912,5cyclictest21749-21awk23:15:102
26223991912,4cyclictest3155-21munin-plugin-st21:50:002
26223991912,4cyclictest14771-21sh21:58:082
26223991911,6cyclictest25203-21cat20:20:182
26223991911,6cyclictest18277-21tune2fs20:05:152
26223991911,6cyclictest14777-21sed22:45:242
26223991911,6cyclictest12358-21awk19:50:212
26223991911,5cyclictest25896-21sed00:30:012
26223991911,5cyclictest24232-21diskstats22:05:152
26223991911,5cyclictest24232-21diskstats22:05:152
26223991911,5cyclictest15152-21uname21:10:232
2622399191,13cyclictest24732-21ssh00:05:162
26223991910,7cyclictest11459-21df_inode23:55:152
2622299199,7cyclictest0-21swapper/100:10:171
2622299193,3cyclictest13484-21diskmemload23:57:261
2622299192,13cyclictest4291-21ssh23:50:101
2622299192,12cyclictest13911-21apt-get21:10:131
2622299191,8cyclictest31796-21ssh21:46:161
26222991912,5cyclictest52850irq/140-enp3s0-22:05:171
26222991912,5cyclictest52850irq/140-enp3s0-22:05:171
26222991911,5cyclictest31251-21mailstats23:45:211
26222991910,6cyclictest31487-21sed19:20:171
26221991914,3cyclictest28772-21acpi20:30:140
2622199191,3cyclictest141rcu_preempt22:50:150
26221991913,4cyclictest32337-21sh23:22:280
26221991913,4cyclictest29248-21sh21:20:270
26221991913,4cyclictest27070-21sh22:30:310
26221991913,4cyclictest16092-21sh21:59:280
26221991912,5cyclictest624-21gdbus20:55:010
26221991912,5cyclictest560-21dbus-daemon22:50:000
26221991912,5cyclictest18078-21cat00:00:130
2622199191,16cyclictest27013-21fschecks_count20:25:150
26221991911,6cyclictest17187-21sed20:00:250
26221991910,7cyclictest5931-21ls22:40:000
26221991910,7cyclictest30045-21awk23:45:120
26221991910,7cyclictest1630-21grep23:00:120
2622199190,2cyclictest141rcu_preempt23:35:390
2622199190,17cyclictest16338-21swap23:10:220
2622499189,7cyclictest24241-21entropy22:05:153
2622499189,7cyclictest24241-21entropy22:05:153
2622499189,6cyclictest9690-21grep19:45:173
2622499189,6cyclictest1258-21idleruntime20:40:153
2622499187,9cyclictest0-21swapper/300:35:043
2622499182,11cyclictest24711-21apt-get00:05:173
26224991813,3cyclictest1657-21python319:25:243
26224991811,5cyclictest20238-21expr20:10:123
26224991810,6cyclictest6654-21cat22:40:133
26224991810,6cyclictest3369-21cat19:30:183
26224991810,6cyclictest3193-21grep20:45:143
26224991810,6cyclictest31480-21expr20:35:173
26224991810,6cyclictest30104-21awk19:15:253
26224991810,6cyclictest26842-21expr20:25:143
26224991810,5cyclictest34-21ksoftirqd/322:50:103
2622399188,6cyclictest53050irq/142-enp3s0-22:00:142
2622399182,11cyclictest28614-21apt-get19:15:112
2622399182,10cyclictest2637-21apt-get19:30:122
2622399181,9cyclictest18247-21df_inode00:00:142
26223991812,4cyclictest4522-21sh21:26:202
26223991811,5cyclictest5295-21expr19:35:132
26223991811,5cyclictest4342-21date23:25:252
26223991811,5cyclictest28976-21expr20:30:162
26223991811,5cyclictest2481-21ssh00:35:322
26223991811,5cyclictest24550-21python323:40:222
26223991811,5cyclictest1962-21awk19:25:262
26223991811,5cyclictest15559-21tail23:10:152
2622399181,12cyclictest31432-21ssh00:10:122
26223991810,5cyclictest4331-21kernelversion21:50:172
26223991810,5cyclictest28-21ksoftirqd/220:55:152
2622299189,6cyclictest23335-21sed23:15:251
2622299188,7cyclictest0-21swapper/119:45:181
2622299187,9cyclictest0-21swapper/123:22:351
2622299184,4cyclictest141rcu_preempt22:00:211
2622299181,7cyclictest1742-21taskset23:00:131
26222991813,3cyclictest48202chrt21:50:221
26222991812,4cyclictest7728-21swap22:40:231
26222991812,4cyclictest31877-21sh22:35:041
26222991811,5cyclictest3699-21cut23:25:191
26222991811,5cyclictest278-21systemd-journal21:00:011
26222991811,5cyclictest19301-21grep20:05:231
26222991811,5cyclictest12455-21seq23:07:541
26222991810,6cyclictest30875-21sed20:35:001
2622199188,7cyclictest4280-21apt-get23:50:100
2622199187,8cyclictest25812-21open_inodes00:05:260
2622199182,11cyclictest454-21apt-get19:25:130
26221991813,3cyclictest9150-21sh22:41:280
26221991813,3cyclictest19612-21sh21:14:210
26221991811,5cyclictest7692-21expr19:40:160
26221991811,5cyclictest7692-21expr19:40:160
26221991811,5cyclictest22119-21find20:15:110
26221991811,5cyclictest18771-21ntp_states20:05:200
26221991810,6cyclictest13660-21cat00:20:180
26221991810,6cyclictest11776-21grep19:50:140
26221991810,6cyclictest10367-21ntpq21:00:180
26221991810,5cyclictest20056-21cat20:10:120
2622499179,6cyclictest5622-21interrupts19:35:173
26224991712,3cyclictest11744-21sh19:50:143
26224991711,4cyclictest24583-21cpuspeed_turbos20:20:133
26224991710,5cyclictest30941-21cat19:20:113
26224991710,5cyclictest13832-21expr19:55:133
2622399179,6cyclictest26743-21expr20:25:142
2622399179,6cyclictest21631-21cat22:50:232
2622399179,6cyclictest13360-21hddtemp_smartct00:20:172
2622399172,11cyclictest15720-21apt-get20:00:122
26223991710,5cyclictest31537-21basename20:35:172
2622299178,6cyclictest1801-21awk20:40:211
2622299177,7cyclictest560-21dbus-daemon20:40:001
2622299171,7cyclictest28136-21ssh21:44:201
26222991712,3cyclictest20084-21/usr/sbin/munin22:50:151
26222991711,4cyclictest12931-21timerwakeupswit21:05:231
2622299171,13cyclictest15817-21tr20:00:141
2622299171,12cyclictest19504-21apt-get00:25:111
2622299171,12cyclictest14989-21apt-get23:10:101
26222991710,5cyclictest7017-21grep20:55:011
26222991710,5cyclictest24726-21expr20:20:141
2622199178,6cyclictest13981-21tune2fs19:55:140
2622199175,3cyclictest141rcu_preempt21:50:110
26221991712,3cyclictest3984-21switchtime20:45:220
26221991712,3cyclictest20619-21open_inodes00:25:220
26221991711,4cyclictest922-21ls00:35:140
26221991711,4cyclictest1257-21idleruntime20:40:160
26221991710,5cyclictest9503-21expr19:45:150
2622399168,6cyclictest12886-21cat21:05:232
26223991611,3cyclictest635-21/usr/sbin/munin20:40:162
2622299169,5cyclictest26739-21grep20:25:141
2622299169,4cyclictest0-21swapper/121:00:081
2622299168,6cyclictest0-21swapper/119:25:131
2622299167,7cyclictest21882-21cron20:15:001
26222991612,2cyclictest2787-21/usr/sbin/munin20:45:121
26222991611,3cyclictest7121-21/usr/sbin/munin19:40:151
26222991611,3cyclictest7121-21/usr/sbin/munin19:40:141
2622199169,5cyclictest3016-21grep19:30:140
2622199168,6cyclictest7930-21cat20:55:180
2622199162,10cyclictest28156-21apt-get19:15:040
2622299157,6cyclictest23619-21grep20:15:261
2622299156,7cyclictest13731-21cstates19:55:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional