You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-17 - 08:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Fri Jul 17, 2026 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
266462222101,97sleep00-21swapper/019:07:550
24669216497,60sleep20-21swapper/219:05:002
265212145110,25sleep10-21swapper/119:06:201
265502143102,29sleep30-21swapper/319:06:423
273732810,5sleep127376-21sh23:45:001
326042720,2chrt0-21swapper/123:00:151
12522690,2sleep30-21swapper/322:12:163
20662550,10sleep012-21ksoftirqd/022:13:050
320032540,2sleep20-21swapper/200:35:502
289662530,2sleep10-21swapper/123:20:481
289662530,2sleep10-21swapper/123:20:481
287102520,2sleep00-21swapper/023:45:190
138682520,2sleep30-21swapper/323:58:563
204402500,1sleep0131rcuc/023:15:140
128012500,2sleep10-21swapper/123:09:451
120682500,1sleep30-21swapper/319:50:113
259732460,2sleep20-21swapper/221:18:382
155792450,5sleep10-21swapper/100:24:411
166212430,2chrt0-21swapper/021:10:570
43222410,1sleep00-21swapper/021:25:540
2708199350,14cyclictest72462fschecks_count23:05:152
2708099343,5cyclictest22-21ksoftirqd/121:45:001
48312320,3sleep228-21ksoftirqd/222:39:322
296392320,1chrt0-21swapper/123:45:321
2708099315,13cyclictest22-21ksoftirqd/121:45:141
2708099312,4cyclictest22-21ksoftirqd/120:05:001
27080993110,7cyclictest22-21ksoftirqd/119:35:011
198642310,2chrt0-21swapper/122:50:191
134242310,10sleep122-21ksoftirqd/122:45:211
308162300,2sleep00-21swapper/022:34:480
2708099303,4cyclictest22-21ksoftirqd/122:05:011
2708099290,7cyclictest22-21ksoftirqd/122:05:241
2708099290,10cyclictest30006-21ssh00:10:231
2707999299,16cyclictest12-21ksoftirqd/022:35:140
2708099289,9cyclictest141rcu_preempt23:40:001
2708099287,7cyclictest22-21ksoftirqd/121:35:151
2708099284,7cyclictest22-21ksoftirqd/119:50:131
2708099283,4cyclictest22-21ksoftirqd/122:30:001
2708099283,4cyclictest22-21ksoftirqd/120:15:001
2708099281,9cyclictest22-21ksoftirqd/121:00:211
2708099280,5cyclictest221ksoftirqd/121:20:231
2707999286,19cyclictest9625-21munin-run21:00:000
27079992818,8cyclictest26662-21ssh23:20:010
27079992818,8cyclictest26662-21ssh23:20:010
2708299272,20cyclictest21135-21ssh23:40:123
2708199275,14cyclictest19480-21ssh00:02:492
2708099277,7cyclictest22-21ksoftirqd/122:20:001
2708099275,9cyclictest22-21ksoftirqd/100:05:261
2708099275,7cyclictest22-21ksoftirqd/119:25:241
2708099274,7cyclictest22-21ksoftirqd/123:30:001
2708099274,6cyclictest22-21ksoftirqd/100:15:171
2708099273,5cyclictest22-21ksoftirqd/100:00:231
2708099272,4cyclictest22-21ksoftirqd/120:25:011
2708099272,4cyclictest22-21ksoftirqd/119:15:001
27080992722,3cyclictest211rcuc/121:53:271
27080992722,3cyclictest211rcuc/121:53:271
27080992722,3cyclictest211rcuc/121:33:161
27080992722,3cyclictest211rcuc/121:25:191
2708099272,10cyclictest22-21ksoftirqd/119:15:161
2707999278,9cyclictest12-21ksoftirqd/021:41:400
72652260,3sleep30-21swapper/321:52:423
72652260,3sleep30-21swapper/321:52:423
43162260,2sleep10-21swapper/123:50:581
274602260,2sleep30-21swapper/323:20:143
274602260,2sleep30-21swapper/323:20:143
27082992610,4cyclictest66050lldpd22:47:003
27081992614,9cyclictest12442-21sh22:45:122
2708199261,23cyclictest0-21swapper/219:10:192
2708099266,8cyclictest22-21ksoftirqd/120:50:001
2708099266,7cyclictest22-21ksoftirqd/123:10:261
2708099263,5cyclictest22-21ksoftirqd/120:05:201
2708099263,4cyclictest22-21ksoftirqd/119:50:011
2708099263,16cyclictest7809-21ssh23:30:121
2708099262,4cyclictest22-21ksoftirqd/122:40:001
2708099262,4cyclictest22-21ksoftirqd/119:45:011
27080992620,4cyclictest211rcuc/100:29:091
2708099261,8cyclictest22-21ksoftirqd/100:30:131
2708099261,7cyclictest22-21ksoftirqd/122:15:011
2708099261,11cyclictest30010-21cut00:35:131
2708299251,1cyclictest0-21swapper/319:10:303
2708199251,14cyclictest0-21swapper/221:25:032
2708099257,8cyclictest22-21ksoftirqd/122:22:121
2708099255,7cyclictest22-21ksoftirqd/120:50:001
2708099254,8cyclictest22-21ksoftirqd/123:55:131
2708099254,7cyclictest22-21ksoftirqd/123:15:151
2708099253,9cyclictest22-21ksoftirqd/121:00:001
2708099253,4cyclictest22-21ksoftirqd/120:35:011
2708099251,9cyclictest22-21ksoftirqd/122:55:131
2708099251,6cyclictest211rcuc/122:40:351
27080992511,3cyclictest22-21ksoftirqd/121:57:321
2708099250,9cyclictest22-21ksoftirqd/121:15:271
2708099250,8cyclictest22-21ksoftirqd/120:20:001
2708099250,7cyclictest22-21ksoftirqd/122:30:011
2708099250,7cyclictest22-21ksoftirqd/121:05:191
2707999252,9cyclictest75-21kswapd000:10:110
27079992515,7cyclictest2465-21python323:25:250
121832250,2sleep20-21swapper/221:32:022
295432241,2sleep30-21swapper/322:09:523
2708199241,16cyclictest18040-21users00:25:252
2708099243,4cyclictest22-21ksoftirqd/120:00:001
2708099241,4cyclictest22-21ksoftirqd/121:15:001
2707999247,8cyclictest12-21ksoftirqd/000:15:110
27079992412,10cyclictest12635-21seq23:57:390
2707999241,11cyclictest0-21swapper/022:17:290
2707999240,1cyclictest0-21swapper/019:10:200
52102230,3sleep2271rcuc/222:15:202
2708299234,6cyclictest0-21swapper/319:20:113
2708299234,4cyclictest66050lldpd23:07:003
27082992312,9cyclictest13301-21ssh21:57:023
2708199231,4cyclictest23362-21ntp_states21:40:202
27081992314,7cyclictest21952-21ls20:10:242
27081992313,7cyclictest29587-21df19:15:152
2708099233,8cyclictest22-21ksoftirqd/120:40:211
2708099233,5cyclictest22-21ksoftirqd/119:20:131
2708099231,6cyclictest22-21ksoftirqd/119:35:231
2707999237,3cyclictest141rcu_preempt21:30:310
27079992316,5cyclictest12-21ksoftirqd/000:40:000
27079992314,6cyclictest21458-21diskstats21:15:150
2708299227,8cyclictest141rcu_preempt20:50:253
27082992216,4cyclictest24678-21sendmail20:20:003
27082992215,5cyclictest34-21ksoftirqd/321:15:003
27082992213,6cyclictest52950irq/141-enp3s0-22:40:133
27082992213,6cyclictest21517-21grep21:15:163
2708299221,12cyclictest141rcu_preempt22:35:233
2708199229,2cyclictest13925-21diskmemload23:53:562
27081992213,7cyclictest7300-21cron19:40:002
27081992212,6cyclictest52750irq/139-enp3s0-23:40:122
27081992212,4cyclictest28-21ksoftirqd/220:00:002
27081992211,7cyclictest7735-21apt-get20:55:112
2708199221,11cyclictest13925-21diskmemload00:34:362
27081992210,4cyclictest28-21ksoftirqd/223:35:262
2708099220,7cyclictest22-21ksoftirqd/120:20:241
2708099220,6cyclictest22-21ksoftirqd/120:35:251
2707999229,11cyclictest12-21ksoftirqd/023:35:010
2707999228,3cyclictest327292grep20:35:230
2707999221,8cyclictest0-21swapper/021:21:520
2707999221,7cyclictest22097-21apt-get00:05:160
27079992216,4cyclictest24087-21grep22:30:000
27079992215,5cyclictest23693-21cstates22:05:150
27079992214,6cyclictest16966-21cat00:00:260
27079992214,6cyclictest12206-21ls21:05:130
27079992213,7cyclictest26892-21ntpq22:55:200
214782220,1sleep20-21swapper/221:39:362
17682220,4sleep20-21swapper/223:00:272
2708299219,3cyclictest34-21ksoftirqd/321:45:273
2708299219,10cyclictest4812-21ssh23:27:253
2708299217,2cyclictest141rcu_preempt20:55:253
2708299216,3cyclictest141rcu_preempt19:35:243
2708299214,2cyclictest141rcu_preempt00:15:273
2708299213,9cyclictest141rcu_preempt23:54:093
2708299213,8cyclictest640-21snmpd00:35:133
2708299213,13cyclictest17836-21ssh22:25:133
2708299212,8cyclictest141rcu_preempt21:20:133
2708299212,3cyclictest13925-21diskmemload00:22:153
27082992114,5cyclictest52950irq/141-enp3s0-23:10:153
27082992114,3cyclictest52950irq/141-enp3s0-22:17:223
27082992113,6cyclictest2065-21ntp_states19:25:203
27082992110,8cyclictest11092-21/usr/sbin/munin22:20:133
2708199212,13cyclictest27783-21apt-get23:45:112
2708199212,10cyclictest15559-21/usr/sbin/munin21:10:212
27081992117,3cyclictest31679-21ssh00:11:412
27081992111,7cyclictest26365-21grep22:30:262
27081992111,2cyclictest13925-21diskmemload00:05:162
2708199210,15cyclictest31554-21/usr/sbin/munin20:35:182
27079992113,6cyclictest3904-21seq23:50:330
27079992113,6cyclictest16731-21grep00:25:150
27079992112,6cyclictest15710-21seq23:10:560
209252210,1sleep30-21swapper/320:10:133
157342210,1chrt0-21swapper/022:23:210
2708299207,9cyclictest52950irq/141-enp3s0-22:55:203
2708299206,9cyclictest52850irq/140-enp3s0-00:25:193
2708299205,3cyclictest141rcu_preempt00:30:143
2708299202,9cyclictest141rcu_preempt21:30:403
2708299202,13cyclictest24949-21chrt20:20:123
2708299202,13cyclictest24727-21sh22:30:123
2708299202,12cyclictest2109-21apt-get21:25:113
2708299201,6cyclictest640-21snmpd00:00:443
27082992012,6cyclictest2868-21seq20:40:483
27082992012,6cyclictest19116-21sh22:50:103
27082992012,6cyclictest10743-21sed19:45:213
27082992012,5cyclictest11286-21grep21:00:253
2708299201,10cyclictest0-21swapper/323:16:413
2708299200,7cyclictest4760-21sed20:45:243
2708199204,6cyclictest0-21swapper/223:20:102
2708199204,6cyclictest0-21swapper/223:20:092
2708199201,9cyclictest13925-21diskmemload23:31:152
2708199201,9cyclictest10271-21hddtemp_smartct00:20:162
2708199201,8cyclictest17747-21ntp_states22:00:212
2708199201,5cyclictest23805-21ps20:15:232
27081992014,4cyclictest31006-21sh21:21:542
27081992012,6cyclictest2529-21grep19:25:242
27081992012,6cyclictest10222-21date23:55:252
2708199201,13cyclictest27307-21chrt20:25:132
2708199201,13cyclictest0-21swapper/223:25:262
2708199201,10cyclictest13925-21diskmemload22:57:562
2708199201,10cyclictest13925-21diskmemload21:57:282
2707999209,3cyclictest237112/usr/sbin/munin20:15:230
2707999208,4cyclictest53050irq/142-enp3s0-23:41:530
2707999208,3cyclictest141rcu_preempt23:35:160
2707999207,3cyclictest12-21ksoftirqd/000:20:130
2707999204,3cyclictest13925-21diskmemload00:34:270
2707999201,5cyclictest0-21swapper/021:46:550
27079992013,5cyclictest18467-21seq20:05:000
27079992013,4cyclictest18502-21sh22:49:460
27079992012,6cyclictest12860-21seq21:56:370
27079992012,5cyclictest18243-21cron20:05:000
27079992011,7cyclictest22303-21seq20:12:150
27079992010,8cyclictest24787-21ssh22:54:220
2707999200,18cyclictest17568-21sed22:00:190
185952201,2sleep20-21swapper/223:13:522
2708299199,8cyclictest21040-21ssh22:03:043
2708299196,9cyclictest52950irq/141-enp3s0-23:35:333
2708299194,2cyclictest141rcu_preempt00:09:183
2708299193,11cyclictest27303-21apt-get20:25:123
2708299192,4cyclictest34-21ksoftirqd/321:36:223
2708299192,13cyclictest7759-21taskset19:40:043
2708299191,8cyclictest10836-21ssh23:31:463
27082991914,3cyclictest13925-21diskmemload23:02:053
27082991911,6cyclictest28159-21sh23:45:163
27082991911,6cyclictest16700-21df_inode20:00:143
27082991910,7cyclictest29869-21awk00:10:223
2708199194,7cyclictest28-21ksoftirqd/222:05:142
2708199192,2cyclictest141rcu_preempt22:50:212
2708199191,9cyclictest13925-21diskmemload22:42:242
2708199191,8cyclictest13925-21diskmemload22:12:412
2708199191,7cyclictest21518-21python323:15:252
2708199191,6cyclictest4028-21irqstats21:50:182
2708199191,6cyclictest4028-21irqstats21:50:182
2708199191,4cyclictest0-21swapper/222:26:062
27081991914,3cyclictest30535-21sendmail_mailst20:30:232
27081991913,4cyclictest6266-21sh00:17:022
27081991912,5cyclictest25770-21ls20:20:202
27081991911,6cyclictest32246-21ls19:20:172
27081991911,6cyclictest18969-21fw_packets20:05:162
27081991911,6cyclictest12969-21ntp_kernel_pll_19:50:202
27081991911,4cyclictest13925-21diskmemload21:45:272
2707999199,6cyclictest53050irq/142-enp3s0-21:35:120
2707999197,8cyclictest31422-21apt-get19:20:100
2707999197,3cyclictest53050irq/142-enp3s0-21:53:180
2707999197,3cyclictest53050irq/142-enp3s0-21:53:170
2707999196,4cyclictest12-21ksoftirqd/023:04:320
2707999192,12cyclictest0-21swapper/023:05:110
27079991913,4cyclictest12-21ksoftirqd/020:30:000
27079991913,3cyclictest10191-21sh22:43:100
27079991911,6cyclictest6575-21mailstats19:35:220
27079991911,6cyclictest4900-21seq19:31:420
27079991911,6cyclictest29792-21forks20:30:170
27079991911,6cyclictest29599-21df19:15:150
27079991911,6cyclictest2075-21awk19:25:200
27079991910,6cyclictest6318-21ls20:50:190
27079991910,6cyclictest26265-21timerwakeupswit20:20:260
2708299189,7cyclictest560-21dbus-daemon20:10:003
2708299181,2cyclictest141rcu_preempt20:35:113
2708299181,2cyclictest141rcu_preempt20:00:003
27082991811,5cyclictest30190-21ls20:30:193
2708299181,15cyclictest24172-21ssh21:40:323
27082991810,6cyclictest4091-21sed19:30:183
2708199182,3cyclictest141rcu_preempt20:40:232
2708199181,7cyclictest8265-21grep19:40:152
2708199181,6cyclictest17525-21python320:00:242
2708199181,3cyclictest0-21swapper/221:00:232
27081991811,5cyclictest4690-21date20:45:242
27081991811,5cyclictest12229-21cat21:05:132
27081991811,5cyclictest11460-21expr22:20:142
27081991810,6cyclictest3468-21expr19:30:122
27081991810,6cyclictest11807-21cron19:50:002
2708199180,6cyclictest6577-21grep20:50:222
2707999189,7cyclictest13887-21seq19:54:170
27079991813,3cyclictest11028-21sendmail_mailst19:45:240
27079991811,5cyclictest15866-21seq19:57:140
27079991810,6cyclictest3268-21date20:45:010
2708299171,4cyclictest0-21swapper/319:15:153
2707999179,6cyclictest9545-21seq19:43:260
2707999178,6cyclictest9856-21seq21:00:030
2707999171,11cyclictest3460-21apt-get20:45:130
2708299157,6cyclictest12178-21tr21:05:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional