You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-24 - 13:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Fri Jul 24, 2026 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
296382217100,97sleep20-21swapper/219:06:142
298872157102,28sleep10-21swapper/119:09:261
299192152105,26sleep30-21swapper/319:09:533
2701221470,1sleep20-21swapper/200:06:062
297872137102,24sleep00-21swapper/019:08:100
1053021100,2sleep20-21swapper/222:17:222
1557021010,3sleep0131rcuc/022:45:180
83732670,2sleep10-21swapper/123:28:241
104762570,2sleep10-21swapper/123:54:371
92962550,1sleep10-21swapper/121:28:061
323862550,2sleep20-21swapper/200:35:122
36572510,1sleep30-21swapper/322:12:093
11772500,1chrt0-21swapper/223:23:032
84232480,1sleep00-21swapper/000:16:360
29212480,2sleep00-21swapper/023:24:500
239062480,1sleep10-21swapper/122:50:531
239062480,1sleep10-21swapper/122:50:521
312632400,3chrt0-21swapper/123:45:171
27672400,2sleep00-21swapper/023:00:150
135302340,2chrt0-21swapper/323:31:513
277032320,2chrt0-21swapper/321:41:233
272942320,9sleep012-21ksoftirqd/020:16:250
163212320,3chrt0-21swapper/123:10:151
3020499313,4cyclictest22-21ksoftirqd/121:45:001
119852310,10sleep228-21ksoftirqd/221:54:472
3020699296,5cyclictest34-21ksoftirqd/300:14:343
3020499292,4cyclictest22-21ksoftirqd/100:00:001
250262290,1sleep10-21swapper/122:28:211
110222290,2chrt0-21swapper/100:19:121
3020699289,10cyclictest34-21ksoftirqd/321:25:123
3020699281,3cyclictest0-21swapper/319:16:383
30205992820,5cyclictest21152-21http22:49:592
3020499288,7cyclictest22-21ksoftirqd/122:10:151
3020499284,6cyclictest22-21ksoftirqd/122:00:011
3020499282,6cyclictest22-21ksoftirqd/100:30:001
3020499282,4cyclictest22-21ksoftirqd/121:15:011
30204992824,3cyclictest211rcuc/122:35:341
30204992811,5cyclictest22-21ksoftirqd/121:35:001
163402280,8sleep228-21ksoftirqd/223:10:152
30205992714,6cyclictest28-21ksoftirqd/221:46:072
30205992713,7cyclictest9731-21rm23:29:482
3020499277,7cyclictest22-21ksoftirqd/122:00:251
3020499276,8cyclictest22-21ksoftirqd/122:35:001
3020499275,8cyclictest22-21ksoftirqd/100:20:121
3020499274,2cyclictest141rcu_preempt23:05:011
3020499272,5cyclictest22-21ksoftirqd/123:25:001
3020499272,4cyclictest22-21ksoftirqd/121:54:591
30204992724,2cyclictest211rcuc/123:32:421
3020499270,7cyclictest26516-1kworker/u9:100:08:171
3020399271,24cyclictest0-21swapper/019:15:230
201912270,1sleep00-21swapper/020:00:200
30205992613,6cyclictest1560-21ssh23:47:192
3020599261,23cyclictest0-21swapper/219:15:202
3020499263,7cyclictest22-21ksoftirqd/122:17:001
30204992614,7cyclictest22-21ksoftirqd/121:50:011
3020499261,1cyclictest0-21swapper/119:16:471
3020499260,9cyclictest22-21ksoftirqd/121:40:011
71402250,2sleep30-21swapper/322:39:353
3020699259,3cyclictest34-21ksoftirqd/320:55:233
3020699257,9cyclictest222492ssh22:25:313
3020699254,4cyclictest34-21ksoftirqd/322:00:193
30206992515,2cyclictest34-21ksoftirqd/323:45:143
30206992511,5cyclictest34-21ksoftirqd/322:49:423
30206992510,3cyclictest317032netstat21:20:233
30206992510,3cyclictest317032netstat21:20:223
3020599258,10cyclictest52750irq/139-enp3s0-23:15:112
3020499258,9cyclictest22-21ksoftirqd/122:58:111
3020499253,8cyclictest22-21ksoftirqd/119:39:591
3020499253,7cyclictest22-21ksoftirqd/120:40:001
3020499253,4cyclictest22-21ksoftirqd/119:35:001
3020499252,6cyclictest22-21ksoftirqd/123:40:431
175962250,4sleep334-21ksoftirqd/300:00:003
3020699242,17cyclictest640-21snmpd19:45:253
3020699242,17cyclictest640-21snmpd19:45:243
3020699241,4cyclictest0-21swapper/321:13:183
30206992413,7cyclictest32417-21apt-get00:35:123
3020599248,4cyclictest28-21ksoftirqd/222:26:152
30205992417,5cyclictest29385-21ssh00:32:212
3020499249,2cyclictest141rcu_preempt00:30:061
3020499248,11cyclictest211rcuc/123:15:141
3020499246,9cyclictest22-21ksoftirqd/121:23:381
3020499246,9cyclictest22-21ksoftirqd/121:23:371
3020499246,10cyclictest22-21ksoftirqd/100:37:441
3020499243,7cyclictest22-21ksoftirqd/123:07:491
3020499243,7cyclictest22-21ksoftirqd/100:10:591
3020499242,15cyclictest14904-21ssh22:45:131
30204992420,3cyclictest211rcuc/121:18:531
30204992419,3cyclictest211rcuc/122:05:301
3020499241,7cyclictest22-21ksoftirqd/100:03:251
3020499241,6cyclictest22-21ksoftirqd/122:42:471
30203992413,7cyclictest28406-21apt-get22:55:120
90632230,1sleep30-21swapper/320:50:203
3020699236,3cyclictest141rcu_preempt00:19:563
3020699233,3cyclictest141rcu_preempt21:50:263
3020699232,4cyclictest0-21swapper/321:04:563
3020699231,5cyclictest0-21swapper/322:21:253
3020699231,4cyclictest0-21swapper/321:48:523
3020599237,3cyclictest31320-21gzip19:10:232
30205992312,6cyclictest28-21ksoftirqd/223:07:102
30205992312,2cyclictest9866-21ssh00:18:032
3020499235,7cyclictest22-21ksoftirqd/120:05:001
3020499233,6cyclictest22-21ksoftirqd/121:10:001
3020499233,6cyclictest22-21ksoftirqd/120:05:141
3020499233,4cyclictest22-21ksoftirqd/122:20:171
3020499232,7cyclictest22-21ksoftirqd/119:30:001
3020499231,7cyclictest22-21ksoftirqd/123:36:201
40802220,2sleep00-21swapper/022:12:350
3020699228,3cyclictest141rcu_preempt23:05:363
3020699226,4cyclictest8908-21fschecks_count19:35:163
3020699223,4cyclictest25954-21/usr/sbin/munin22:05:263
30206992219,2cyclictest23345-21tr23:40:003
30206992212,3cyclictest34-21ksoftirqd/321:35:193
30206992212,3cyclictest34-21ksoftirqd/320:40:203
3020599227,7cyclictest3941-21ssh21:24:252
3020599227,7cyclictest3941-21ssh21:24:242
3020599227,12cyclictest560-21dbus-daemon23:05:002
3020599221,6cyclictest0-21swapper/222:10:542
30205992213,6cyclictest27054-21ntpq22:05:212
3020599221,2cyclictest0-21swapper/220:10:212
30205992212,5cyclictest28-21ksoftirqd/223:40:222
30205992212,5cyclictest28-21ksoftirqd/221:14:402
30205992211,6cyclictest28-21ksoftirqd/221:44:532
30205992210,3cyclictest5831-21/usr/sbin/munin21:25:182
3020499223,6cyclictest22-21ksoftirqd/119:45:251
3020499223,6cyclictest22-21ksoftirqd/119:45:251
3020499222,4cyclictest22-21ksoftirqd/121:00:161
3020499221,8cyclictest22-21ksoftirqd/119:40:141
3020399223,13cyclictest26005-21ssh22:05:120
3020399221,8cyclictest636-21runrttasks00:03:060
30203992213,7cyclictest28306-21seq23:19:290
30203992213,6cyclictest22341-21ntp_states20:05:200
153782220,1sleep30-21swapper/321:32:283
3020699218,4cyclictest34-21ksoftirqd/300:30:403
3020699215,2cyclictest34-21ksoftirqd/323:10:193
3020699213,15cyclictest25109-21pmu-power21:15:233
30206992118,2cyclictest341ksoftirqd/323:45:013
3020699211,5cyclictest0-21swapper/323:03:233
3020699211,4cyclictest0-21swapper/322:58:383
30206992112,6cyclictest3054-21ntp_states19:20:223
30206992112,6cyclictest18575-21awk19:55:263
30206992111,8cyclictest560-21dbus-daemon22:35:003
30206992110,3cyclictest34-21ksoftirqd/300:00:263
3020699210,4cyclictest331rcuc/300:05:253
3020599219,7cyclictest141rcu_preempt20:20:212
3020599217,7cyclictest14968-21sshd00:21:252
3020599216,3cyclictest30323-21cat20:25:132
3020599212,11cyclictest141rcu_preempt21:38:162
3020599211,6cyclictest10913-21gltestperf23:30:152
30205992111,7cyclictest5605-21needreboot23:50:202
30205992111,7cyclictest28871-21tune2fs22:55:172
30205992111,6cyclictest2139-21apt-get19:20:142
30205992110,6cyclictest28-21ksoftirqd/222:38:462
3020599210,8cyclictest11970-21/usr/sbin/munin21:30:142
3020499213,7cyclictest22-21ksoftirqd/119:50:251
3020499213,6cyclictest22-21ksoftirqd/120:30:241
3020499213,4cyclictest22-21ksoftirqd/120:55:131
3020499213,3cyclictest22-21ksoftirqd/120:20:121
3020499213,2cyclictest141rcu_preempt20:30:001
3020499212,4cyclictest22-21ksoftirqd/119:20:151
3020499210,6cyclictest22-21ksoftirqd/119:55:221
30203992115,4cyclictest12-21ksoftirqd/022:20:000
30203992115,4cyclictest12-21ksoftirqd/021:59:590
30203992113,6cyclictest2897-21memory19:20:200
30203992112,6cyclictest27053-21diskmemload22:30:110
30203992111,7cyclictest13953-21diskstats22:20:150
3020399211,11cyclictest12736-1kworker/u9:222:26:150
30203992110,8cyclictest15954-21apt-get23:10:120
130722210,9sleep334-21ksoftirqd/322:43:353
3020699208,3cyclictest34-21ksoftirqd/320:00:253
3020699202,3cyclictest141rcu_preempt21:55:193
3020699202,3cyclictest141rcu_preempt00:20:583
3020699202,14cyclictest29307-21ssh23:20:143
30206992013,5cyclictest28463-21if_enp3s020:20:153
30206992013,4cyclictest7501-21tail22:15:183
30206992010,7cyclictest560-21dbus-daemon00:25:003
3020599209,6cyclictest28-21ksoftirqd/222:32:092
3020599209,3cyclictest28-21ksoftirqd/200:00:232
3020599208,3cyclictest0-21swapper/223:35:152
3020599202,15cyclictest13719-21tr22:20:142
30205992015,3cyclictest21534-21sh22:50:012
30205992015,3cyclictest21534-21sh22:50:002
30205992015,3cyclictest20662-21timerwakeupswit00:25:262
30205992014,4cyclictest9394-21sendmail_mailst20:50:222
30205992014,4cyclictest1876-21cpu20:35:132
3020599201,2cyclictest0-21swapper/222:41:502
3020499206,7cyclictest22-21ksoftirqd/120:16:111
3020499202,5cyclictest22-21ksoftirqd/120:55:011
3020499200,8cyclictest22-21ksoftirqd/119:10:161
3020399205,7cyclictest141rcu_preempt22:51:060
3020399205,7cyclictest141rcu_preempt22:51:060
3020399202,16cyclictest8289-21ssh21:51:000
3020399201,9cyclictest14710-21sleep21:05:000
30203992013,5cyclictest17062-21seq00:23:330
30203992012,6cyclictest5139-21seq22:37:300
30203992012,6cyclictest4953-21seq21:49:230
30203992012,6cyclictest4594-21hwlatdetect20:40:170
30203992012,6cyclictest30901-21cat21:20:150
30203992012,6cyclictest30901-21cat21:20:150
30203992012,6cyclictest26104-21ntp_states21:40:220
30203992011,7cyclictest6667-21cat19:30:120
30203992010,8cyclictest12-21ksoftirqd/021:30:460
30203992010,7cyclictest5369-21entropy21:25:150
30203992010,7cyclictest26136-21df_inode00:30:130
3020699199,8cyclictest34-21ksoftirqd/323:25:103
3020699199,8cyclictest22149-21ssh22:50:163
3020699199,8cyclictest22149-21ssh22:50:153
3020699198,4cyclictest34-21ksoftirqd/323:15:073
3020699197,4cyclictest141rcu_preempt20:25:123
3020699191,7cyclictest5011-21df_inode23:50:163
3020599199,7cyclictest10746-21grep20:55:142
3020599199,5cyclictest14987-21apt-get19:50:132
3020599195,6cyclictest15799-21sh23:58:172
3020599191,8cyclictest20362-21ntp_states22:00:212
3020599191,6cyclictest5908-21cron20:45:002
30205991914,3cyclictest8527-21/usr/sbin/munin19:35:222
30205991913,3cyclictest28-21ksoftirqd/220:30:122
30205991911,6cyclictest7731-21seq20:46:032
30205991911,6cyclictest22965-21seq20:05:432
30205991911,6cyclictest14930-21cat21:05:122
30205991911,6cyclictest12748-21ls21:55:132
30205991911,6cyclictest11026-21expr19:40:142
30205991911,5cyclictest19639-21forks20:00:152
30205991910,6cyclictest7623-21cat19:30:242
3020599190,9cyclictest28-21ksoftirqd/200:12:012
30204991915,3cyclictest211rcuc/120:40:041
3020499191,4cyclictest22-21ksoftirqd/120:10:131
30204991911,6cyclictest7445-21wc20:45:261
3020399199,7cyclictest52950irq/141-enp3s0-21:15:250
30203991913,4cyclictest22628-21sh21:38:020
30203991913,4cyclictest22371-21sh21:13:480
30203991913,4cyclictest12-21ksoftirqd/023:55:000
30203991912,5cyclictest31963-21unixbench-2d23:45:250
30203991912,5cyclictest21257-21seq22:00:390
30203991912,5cyclictest1001-21ntpq00:35:210
30203991912,4cyclictest20832-21unixbench_singl00:25:270
30203991911,6cyclictest9785-21expr19:35:230
30203991911,6cyclictest10438-21sed23:05:230
30203991911,5cyclictest9625-21ls20:50:240
30203991910,6cyclictest11599-21seq20:55:210
3020699188,3cyclictest34-21ksoftirqd/320:05:133
3020699188,3cyclictest34-21ksoftirqd/319:30:123
3020699182,11cyclictest4275-21apt-get19:25:143
3020699181,5cyclictest0-21swapper/319:10:173
30206991812,3cyclictest34-21ksoftirqd/320:10:163
3020599182,11cyclictest12849-21apt-get19:45:132
3020599182,11cyclictest12849-21apt-get19:45:122
30205991812,4cyclictest27637-21sh21:17:242
30205991810,6cyclictest6366-21date19:30:002
30205991810,6cyclictest26950-21expr20:15:232
30205991810,6cyclictest13148-21entropy21:00:152
3020399189,7cyclictest23668-21expr20:10:120
3020399189,7cyclictest11918-21smartctl23:55:170
3020399189,6cyclictest2380-21grep20:35:150
3020399183,9cyclictest7573-21apt-get22:40:030
30203991813,3cyclictest5443-21swap19:25:240
30203991813,3cyclictest3778-21cpuspeed_turbos23:25:130
30203991813,3cyclictest27847-21sh23:42:480
30203991813,3cyclictest26482-21sh00:05:330
30203991813,3cyclictest10260-21sh23:30:050
30203991812,4cyclictest23139-21sh23:39:510
30203991812,4cyclictest12-21ksoftirqd/019:15:000
30203991811,5cyclictest31672-21seq20:27:200
30203991811,5cyclictest28655-21ls20:20:160
30203991811,5cyclictest15496-21expr21:05:160
3020399181,11cyclictest31310-21apt-get00:10:130
30203991810,6cyclictest32337-21expr20:30:120
3020699177,8cyclictest560-21dbus-daemon20:35:003
3020699177,2cyclictest34-21ksoftirqd/320:35:213
3020699171,4cyclictest0-21swapper/319:40:153
30206991710,4cyclictest34-21ksoftirqd/319:50:143
3020699170,3cyclictest141rcu_preempt20:15:113
3020599172,10cyclictest17128-21apt-get19:55:142
3020399179,6cyclictest6521-21fschecks_count20:45:170
3020399179,6cyclictest15887-21ntpq19:50:200
3020399179,5cyclictest10696-21acpi19:40:110
30203991711,4cyclictest17550-21fschecks_time19:55:160
30203991710,5cyclictest13021-21cat19:45:130
30203991710,5cyclictest13021-21cat19:45:120
3020699167,7cyclictest7072-21awk20:45:223
30206991610,4cyclictest16254-21sed21:05:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional