You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-14 - 08:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackaslot8s.osadl.org (updated Tue Jul 14, 2026 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32632181146,24sleep00-21swapper/019:09:360
32632181146,24sleep00-21swapper/019:09:350
228021610,1sleep10-21swapper/122:30:241
3271214799,40sleep30-21swapper/319:09:433
3271214799,40sleep30-21swapper/319:09:423
30292145112,22sleep20-21swapper/219:06:332
30292145112,22sleep20-21swapper/219:06:332
32692140104,25sleep10-21swapper/119:09:411
32692140104,25sleep10-21swapper/119:09:401
225892750,2sleep30-21swapper/321:33:303
115052600,2sleep10-21swapper/123:50:141
252822590,3sleep3331rcuc/322:48:083
125432570,1sleep00-21swapper/022:14:580
74822560,3sleep27475-21ssh22:35:102
278002560,2sleep10-21swapper/123:38:021
19582490,2chrt0-21swapper/021:17:550
316382480,1sleep30-21swapper/322:04:573
302872480,1sleep30-21swapper/323:40:123
173292480,3chrt0-21swapper/321:53:583
78682470,1sleep20-21swapper/200:10:542
299952470,2sleep1211rcuc/123:15:231
252822470,3sleep00-21swapper/000:24:590
50422430,1sleep10-21swapper/100:09:481
72642420,1sleep10-21swapper/123:22:451
169712360,2sleep00-21swapper/021:29:350
3567993519,10cyclictest53050irq/142-enp3s0-22:40:111
207462350,9sleep334-21ksoftirqd/323:56:383
3568993212,11cyclictest28-21ksoftirqd/223:54:152
3568993211,12cyclictest28-21ksoftirqd/200:00:442
66142300,2sleep20-21swapper/223:22:062
356899309,11cyclictest28-21ksoftirqd/200:34:252
3568993013,4cyclictest28-21ksoftirqd/200:25:292
356899299,11cyclictest28-21ksoftirqd/222:57:282
356899299,11cyclictest28-21ksoftirqd/221:42:542
3568992910,7cyclictest28-21ksoftirqd/223:01:002
356899288,8cyclictest28-21ksoftirqd/222:45:432
356899288,10cyclictest28-21ksoftirqd/221:14:072
356899287,12cyclictest28-21ksoftirqd/222:29:542
356899287,10cyclictest28-21ksoftirqd/222:43:212
356899281,7cyclictest28-21ksoftirqd/221:35:222
3568992812,7cyclictest28-21ksoftirqd/200:05:392
3568992810,8cyclictest28-21ksoftirqd/221:54:142
3568992810,7cyclictest28-21ksoftirqd/200:36:042
78292270,1sleep00-21swapper/021:22:050
356899278,7cyclictest28-21ksoftirqd/223:29:162
356899277,8cyclictest28-21ksoftirqd/223:48:272
356899277,11cyclictest28-21ksoftirqd/222:15:292
3568992719,6cyclictest1838-21ssh22:05:502
3568992710,6cyclictest28-21ksoftirqd/222:23:102
3568992710,6cyclictest28-21ksoftirqd/220:45:152
207572270,3sleep1221ksoftirqd/119:50:001
357099262,16cyclictest640-21snmpd21:25:133
357099261,23cyclictest0-21swapper/319:11:043
356899267,8cyclictest28-21ksoftirqd/223:10:362
356899267,11cyclictest28-21ksoftirqd/223:44:002
356899266,10cyclictest28-21ksoftirqd/223:38:072
356899264,7cyclictest28-21ksoftirqd/223:55:172
3568992610,7cyclictest141rcu_preempt21:25:032
3568992610,6cyclictest28-21ksoftirqd/222:34:442
3568992610,6cyclictest28-21ksoftirqd/220:16:132
3568992610,6cyclictest28-21ksoftirqd/200:23:012
356699267,16cyclictest141rcu_preempt22:55:150
3566992624,1cyclictest0-21swapper/019:11:050
105262260,2chrt10528-21tr20:40:173
357099252,16cyclictest13257-21ssh22:15:133
357099252,11cyclictest31209-21ssh21:40:123
356899259,7cyclictest28-21ksoftirqd/223:15:022
356899259,7cyclictest28-21ksoftirqd/221:21:512
356899259,6cyclictest28-21ksoftirqd/221:34:482
356899259,5cyclictest28-21ksoftirqd/222:10:012
356899258,7cyclictest28-21ksoftirqd/222:55:002
356899257,10cyclictest28-21ksoftirqd/222:00:192
356899255,11cyclictest28-21ksoftirqd/223:33:502
356899251,8cyclictest28-21ksoftirqd/221:15:012
356899251,1cyclictest0-21swapper/219:11:042
3568992510,7cyclictest28-21ksoftirqd/221:56:312
3568992510,7cyclictest28-21ksoftirqd/200:15:012
3568992510,6cyclictest28-21ksoftirqd/221:46:412
3567992523,1cyclictest0-21swapper/119:11:001
356699259,3cyclictest0-21swapper/022:35:160
3566992517,5cyclictest21617-21grep22:20:260
259562250,1sleep30-21swapper/322:24:473
356899249,6cyclictest28-21ksoftirqd/223:08:332
356899248,9cyclictest28-21ksoftirqd/220:43:012
356899242,9cyclictest28-21ksoftirqd/220:05:222
356899242,15cyclictest14136-21runrttasks20:50:132
3568992417,5cyclictest28-21ksoftirqd/220:39:592
356899241,6cyclictest10054-21apt-get19:25:132
356799246,3cyclictest22-21ksoftirqd/121:40:161
3567992410,5cyclictest8463-21tr22:35:191
356699247,9cyclictest12-21ksoftirqd/000:35:220
356699242,16cyclictest3087-21ssh23:20:030
275622240,2sleep334-21ksoftirqd/320:05:153
261932240,10sleep012-21ksoftirqd/022:00:170
226582240,3sleep022638-21ssh23:10:160
151732240,1sleep30-21swapper/323:52:453
93832230,9sleep334-21ksoftirqd/323:48:393
3570992313,5cyclictest0-21swapper/323:30:023
356899236,5cyclictest28-21ksoftirqd/221:00:222
356899235,10cyclictest28-21ksoftirqd/221:09:312
3568992314,3cyclictest28-21ksoftirqd/220:20:242
356899230,6cyclictest5559-21apt-get20:30:122
356799231,9cyclictest66050lldpd22:23:461
3566992312,7cyclictest75-21kswapd021:45:120
254602230,2chrt0-21swapper/120:00:151
357099228,10cyclictest9085-21ssh22:35:253
3570992217,3cyclictest657-21sshd21:18:133
356899227,6cyclictest28-21ksoftirqd/220:55:162
356899227,6cyclictest28-21ksoftirqd/220:29:192
356899227,5cyclictest28-21ksoftirqd/219:46:262
356899224,10cyclictest28-21ksoftirqd/219:18:402
356799229,11cyclictest22-21ksoftirqd/100:20:001
3567992214,6cyclictest9471-21seq19:20:551
356699229,8cyclictest26125-21kworker/u8:223:00:120
356699221,9cyclictest22822xargs22:30:240
356699221,17cyclictest24484-21apt-get21:35:120
3566992211,5cyclictest21823-21seq21:32:440
3566992210,3cyclictest141rcu_preempt22:00:000
11002220,1sleep00-21swapper/023:42:010
60122210,11sleep012-21ksoftirqd/020:30:160
357099219,3cyclictest15749-21df_inode23:05:143
357099219,10cyclictest1913-21sh00:30:433
357099212,13cyclictest640-21snmpd20:45:133
357099211,9cyclictest147072fschecks_time22:40:153
3570992113,6cyclictest17650-21awk20:55:243
3570992113,5cyclictest32524-21ntp_offset00:05:233
3570992112,7cyclictest7761-21switchtime22:10:243
3570992112,6cyclictest2251-21ls20:20:213
356899219,10cyclictest28-21ksoftirqd/220:05:002
356899212,6cyclictest28-21ksoftirqd/219:52:132
356899212,6cyclictest28-21ksoftirqd/219:52:132
356899212,3cyclictest141rcu_preempt19:30:122
3568992114,5cyclictest28-21ksoftirqd/219:45:002
3568992110,3cyclictest141rcu_preempt19:55:412
356799219,8cyclictest14334-21apt-get19:35:121
3567992115,4cyclictest278-21systemd-journal22:15:001
3567992113,6cyclictest29851-21fw_packets20:10:131
3567992112,7cyclictest21756-21seq23:57:391
356799211,12cyclictest0-21swapper/100:20:161
356799210,7cyclictest23416-21tr21:10:121
356699218,4cyclictest141rcu_preempt20:05:010
356699215,6cyclictest11495-1kworker/u9:200:17:010
356699214,4cyclictest0-21swapper/000:07:090
356699213,8cyclictest141rcu_preempt22:25:210
3566992113,6cyclictest16310-21ls23:05:190
3566992113,6cyclictest15868-21seq19:35:390
3566992112,6cyclictest9158-21grep20:35:240
356699210,7cyclictest16055-21latency_hist20:55:010
103692211,2sleep30-21swapper/321:24:423
88112200,2sleep30-21swapper/300:11:513
3570992015,3cyclictest16631-21cut19:40:143
3570992012,6cyclictest8600-21ntpq20:35:193
3570992012,6cyclictest14133-21apt-get20:50:123
3570992010,7cyclictest8247-21df_inode19:20:143
3570992010,6cyclictest52950irq/141-enp3s0-23:10:113
356899202,7cyclictest28-21ksoftirqd/219:20:162
356799209,8cyclictest7492-21kworker/u8:023:25:261
356799209,8cyclictest4499-21apt-get21:20:141
356799209,8cyclictest18141-21df21:30:121
356799201,5cyclictest0-21swapper/100:26:071
3567992014,4cyclictest28702-21sh22:01:591
3567992014,4cyclictest21527-21sh21:56:291
3567992013,5cyclictest21541-21seq22:45:161
3567992013,5cyclictest12756-21memory20:45:201
3567992013,4cyclictest24560-21cpu21:35:131
3567992012,6cyclictest9913-21awk20:40:131
3567992012,6cyclictest9292-21seq20:36:221
3567992012,6cyclictest5217-21seq22:09:161
3567992012,6cyclictest20423-21seq23:08:471
3567992012,6cyclictest16878-21tune2fs20:55:161
3567992012,5cyclictest18977-21diskstats21:00:161
3567992011,7cyclictest19636-21date22:20:011
3567992011,7cyclictest16559-21cat19:40:141
3567992011,7cyclictest11136-21cat21:25:121
356699206,5cyclictest12-21ksoftirqd/022:45:210
356699204,10cyclictest5680-21ssh00:10:110
356699203,14cyclictest6023-21grep23:45:250
3566992014,4cyclictest12-21ksoftirqd/000:00:010
3566992014,3cyclictest6152-21gpgv00:35:000
3566992013,5cyclictest30508-21mailstats20:10:210
3566992012,6cyclictest9365-21grep19:20:250
3566992012,6cyclictest29899-21sed23:40:000
3566992012,6cyclictest29852-21sed21:15:000
3566992012,6cyclictest17327-21fw_packets23:30:150
3566992012,6cyclictest10093-21cat20:40:140
357099199,7cyclictest27646-21ntp_states22:25:213
357099198,8cyclictest20818-21apt-get19:50:143
357099198,8cyclictest20818-21apt-get19:50:143
357099192,13cyclictest28716-21apt-get23:15:133
357099192,11cyclictest18680-21apt-get19:45:133
357099191,8cyclictest24368-21taskset23:35:193
3570991913,4cyclictest28463-21sh21:37:423
3570991913,4cyclictest10098-21sh00:37:053
3570991912,5cyclictest29576-21diskmemload22:50:423
3570991912,4cyclictest22666-21diskmemload23:20:023
3570991911,6cyclictest4727-21date20:25:253
3570991911,6cyclictest369-21date20:15:263
3570991911,6cyclictest24227-21seq00:24:003
3570991911,6cyclictest14551-21expr19:35:143
3570991910,7cyclictest6123-21smartctl20:30:173
356899190,7cyclictest28-21ksoftirqd/220:10:132
356799192,12cyclictest1265-21apt-get20:20:121
3567991914,3cyclictest7552-21sh21:45:471
3567991914,3cyclictest31698-21timerwakeupswit23:40:241
3567991913,4cyclictest616-21interrupts00:30:171
3567991913,4cyclictest18902-21sh23:30:481
3567991912,5cyclictest8808-21seq23:48:041
3567991912,5cyclictest5612-21awk20:30:131
3567991912,5cyclictest32226-21sh22:29:251
3567991912,5cyclictest24668-21seq19:57:581
3567991912,5cyclictest14579-21seq21:51:091
3567991911,6cyclictest7794-21ntpq00:35:211
3567991911,6cyclictest5858-21cut00:10:141
3567991911,6cyclictest11813-21seq19:28:321
3567991910,6cyclictest13085-21ntp_kernel_pll_19:30:211
356699192,4cyclictest12-21ksoftirqd/022:44:520
356699192,11cyclictest18153-21apt-get19:42:020
3566991914,3cyclictest636-21runrttasks23:26:260
3566991912,5cyclictest18381-21ls21:00:010
3566991910,7cyclictest13619-21grep21:50:250
357099189,7cyclictest18882-21cat21:55:143
357099189,6cyclictest20788-21cpu21:05:133
357099188,8cyclictest7162-21sed21:45:263
357099182,12cyclictest843-21grep22:05:223
357099182,11cyclictest23286-21apt-get21:10:113
3570991813,3cyclictest12139-21sh23:25:563
3570991813,3cyclictest10934-21netstat19:25:213
3570991812,4cyclictest28170-21sh00:26:133
3570991811,5cyclictest7360-21seq19:16:333
3570991811,5cyclictest13066-21seq00:15:173
3570991811,4cyclictest1646-21sh22:55:033
3570991810,6cyclictest560-21dbus-daemon20:05:003
3570991810,6cyclictest1120-21expr22:30:133
357099180,7cyclictest24567-21ssh00:00:123
356899181,5cyclictest0-21swapper/219:35:122
356799189,7cyclictest22105-21cat21:05:251
356799188,7cyclictest20806-21cat19:50:121
356799188,7cyclictest20806-21cat19:50:111
356799181,8cyclictest31176-21ssh22:52:211
3567991814,2cyclictest27228-21/usr/sbin/munin20:05:281
3567991813,3cyclictest30124-21sh21:15:101
3567991813,3cyclictest26223-21unixbench_singl00:00:261
3567991813,3cyclictest23059-21netstat23:10:201
356799181,12cyclictest8506-21apt-get23:00:121
3567991810,6cyclictest6685-21awk19:15:211
3567991810,6cyclictest3129-21readlink22:55:241
356699184,12cyclictest141rcu_preempt23:16:070
356699182,11cyclictest27569-21apt-get22:50:130
356699181,6cyclictest0-21swapper/022:19:510
356699181,6cyclictest0-21swapper/022:06:250
3566991813,3cyclictest21104-21df21:05:140
3566991811,5cyclictest14915-21seq23:52:290
3566991811,5cyclictest13125-21awk19:30:210
3566991811,5cyclictest12-21ksoftirqd/000:02:410
3566991811,5cyclictest11238-21switchtime19:25:250
3566991810,6cyclictest3981-21smartctl20:25:170
3566991810,6cyclictest20881-21sort19:50:140
3566991810,6cyclictest20881-21sort19:50:130
11352180,2sleep00-21swapper/020:20:010
3570991712,3cyclictest10989-21sh23:01:113
3570991710,5cyclictest31355-21sed20:15:013
3570991710,5cyclictest19361-21grep21:00:183
3570991710,5cyclictest13662-21grep19:30:273
356799179,6cyclictest32333-21cat20:15:201
356799179,6cyclictest14436-21expr20:50:151
356699179,5cyclictest24277-21timerwakeupswit19:55:260
356699179,5cyclictest20063-21grep21:00:270
356699172,11cyclictest25676-21apt-get00:25:140
356699171,7cyclictest11971-21/usr/sbin/munin20:45:230
356699171,12cyclictest31258-21ssh21:40:130
3566991710,4cyclictest19529-21head19:45:200
356799169,5cyclictest4331-21cat20:25:211
3566991610,4cyclictest27720-21gltestperf20:05:170
357099151,9cyclictest22959-21apt-get19:55:153
356699157,6cyclictest5728-21/usr/sbin/munin19:15:190
356699153,9cyclictest640-21snmpd20:20:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional