You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 13:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Tue Jul 21, 2026 00:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7154217598,25sleep10-21swapper/119:06:551
71262174101,29sleep00-21swapper/019:06:340
71762162111,28sleep30-21swapper/319:07:133
73672158102,29sleep20-21swapper/219:09:412
694321200,3sleep10-21swapper/123:18:551
247902690,2sleep00-21swapper/023:31:420
321012650,2sleep30-21swapper/300:01:073
101682620,1sleep30-21swapper/323:45:173
266422530,2sleep20-21swapper/222:20:252
251882510,1sleep30-21swapper/322:44:223
197922510,1sleep20-21swapper/223:04:362
126692510,2sleep00-21swapper/021:22:280
34562480,1sleep00-21swapper/021:40:140
273392480,1sleep20-21swapper/223:58:172
209752480,1sleep30-21swapper/323:53:323
34122470,2sleep30-21swapper/322:04:003
34122470,2sleep30-21swapper/322:04:003
230492470,1sleep30-21swapper/322:18:173
69412460,2sleep00-21swapper/000:30:490
18862350,7sleep012-21ksoftirqd/000:03:210
133472350,5sleep334-21ksoftirqd/322:59:523
121912330,1sleep10-21swapper/121:22:011
192062300,2chrt131rcuc/022:15:180
7667992912,4cyclictest34-21ksoftirqd/323:34:053
7667992912,4cyclictest34-21ksoftirqd/321:19:453
7667992910,4cyclictest34-21ksoftirqd/300:39:083
7667992812,4cyclictest34-21ksoftirqd/322:30:023
7667992810,4cyclictest34-21ksoftirqd/322:08:463
316902280,2sleep20-21swapper/200:25:212
27822280,2sleep20-21swapper/222:26:592
263072280,2sleep00-21swapper/019:50:230
766799279,8cyclictest141rcu_preempt23:15:273
766799279,4cyclictest34-21ksoftirqd/323:14:383
766799271,2cyclictest141rcu_preempt21:25:233
7666992718,7cyclictest29222-21ssh21:59:092
766599271,24cyclictest0-21swapper/119:16:591
766499270,25cyclictest0-21swapper/019:18:040
306982270,9sleep122-21ksoftirqd/122:24:131
219842270,2chrt0-21swapper/123:54:341
766799269,5cyclictest34-21ksoftirqd/321:49:073
766799269,4cyclictest34-21ksoftirqd/300:12:333
766799268,5cyclictest34-21ksoftirqd/300:25:153
7667992620,4cyclictest34-21ksoftirqd/300:20:003
7667992611,4cyclictest34-21ksoftirqd/321:45:003
7667992610,3cyclictest141rcu_preempt21:58:543
7667992610,3cyclictest141rcu_preempt00:05:253
766699261,23cyclictest0-21swapper/219:19:062
766499264,11cyclictest12-21ksoftirqd/023:47:180
766499261,6cyclictest12-21ksoftirqd/021:50:000
766499261,1cyclictest0-21swapper/019:20:080
7667992510,4cyclictest34-21ksoftirqd/322:46:573
7667992510,4cyclictest34-21ksoftirqd/321:10:333
766699255,11cyclictest9132-21cstates21:20:152
766699250,23cyclictest0-21swapper/219:20:032
7665992523,1cyclictest0-21swapper/119:20:081
766499257,7cyclictest12-21ksoftirqd/022:37:300
766499254,11cyclictest12-21ksoftirqd/023:36:130
766499252,11cyclictest12-21ksoftirqd/022:34:500
7664992516,2cyclictest27850-21ssh00:22:390
7664992516,2cyclictest27850-21ssh00:22:380
175062250,2chrt0-21swapper/123:26:051
766799249,6cyclictest141rcu_preempt23:41:533
766799249,6cyclictest141rcu_preempt21:51:513
766799247,5cyclictest34-21ksoftirqd/322:13:353
766799247,4cyclictest34-21ksoftirqd/322:22:183
766799245,14cyclictest141rcu_preempt20:45:223
7667992417,5cyclictest34-21ksoftirqd/300:35:003
766799241,1cyclictest0-21swapper/319:17:313
7667992410,3cyclictest141rcu_preempt21:38:053
7665992411,8cyclictest30609-21ssh00:25:121
766499245,10cyclictest12-21ksoftirqd/023:52:290
7664992415,5cyclictest3419-21sed23:40:150
7664992412,4cyclictest12-21ksoftirqd/019:30:000
7664992411,6cyclictest24760-21kworker/u8:121:19:290
44662240,10sleep228-21ksoftirqd/220:15:212
766799239,7cyclictest141rcu_preempt20:10:073
766799238,6cyclictest141rcu_preempt23:36:053
766799238,4cyclictest34-21ksoftirqd/321:23:053
766799238,3cyclictest141rcu_preempt23:05:193
766799235,2cyclictest141rcu_preempt23:23:053
766799232,15cyclictest510-21ssh22:50:123
7667992317,4cyclictest34-21ksoftirqd/319:15:003
7667992313,7cyclictest14317-21apt20:40:103
7667992310,8cyclictest15167-21ssh23:25:133
7666992311,7cyclictest52750irq/139-enp3s0-21:10:142
766499236,7cyclictest12-21ksoftirqd/021:13:110
766499235,8cyclictest141rcu_preempt22:51:270
766499235,11cyclictest20379-1kworker/u9:222:04:590
766499235,11cyclictest20379-1kworker/u9:222:04:590
220642230,2sleep2271rcuc/221:30:002
172612230,5sleep1141rcu_preempt21:50:141
154622230,2sleep30-21swapper/322:36:203
114072230,3sleep10-21swapper/122:10:001
766799229,6cyclictest141rcu_preempt19:20:403
766799228,4cyclictest34-21ksoftirqd/300:22:023
766799228,4cyclictest34-21ksoftirqd/300:22:013
766799222,14cyclictest14010-21ssh23:00:143
7667992215,5cyclictest34-21ksoftirqd/322:34:593
7667992212,3cyclictest34-21ksoftirqd/321:30:333
766699221,5cyclictest12123-21ssh21:45:272
766699221,13cyclictest24860-21ls00:20:202
766699221,13cyclictest24860-21ls00:20:202
7665992214,6cyclictest22415-21seq20:57:331
7665992214,5cyclictest21769-21/usr/sbin/munin23:30:181
7665992212,7cyclictest29896-21df20:00:131
7665992212,7cyclictest17963-21cron00:40:001
7665992211,7cyclictest53050irq/142-enp3s0-21:15:121
766499226,9cyclictest12-21ksoftirqd/023:08:490
766499225,4cyclictest12-21ksoftirqd/000:18:180
766499224,8cyclictest12-21ksoftirqd/021:58:240
766499222,10cyclictest0-21swapper/023:20:590
7664992211,7cyclictest560-21dbus-daemon22:55:240
766499221,10cyclictest141rcu_preempt21:34:060
7664992210,2cyclictest12-21ksoftirqd/022:12:160
766499220,12cyclictest26795-21tune2fs22:45:170
766799218,2cyclictest141rcu_preempt20:19:343
766799216,7cyclictest141rcu_preempt23:56:543
766799216,3cyclictest141rcu_preempt20:55:143
7667992111,3cyclictest34-21ksoftirqd/320:00:183
7666992113,6cyclictest24581-21seq21:02:392
7666992113,6cyclictest10968-21ls23:45:252
7666992112,7cyclictest13518-21basename23:00:002
7666992112,3cyclictest20705-21sh23:05:052
766699210,8cyclictest66050lldpd22:16:072
7665992112,7cyclictest24429-21seq21:00:411
766599211,14cyclictest29705-21ssh00:00:111
766499219,3cyclictest32316-21kworker/u8:200:25:340
766499219,3cyclictest12-21ksoftirqd/023:17:270
766499215,7cyclictest12-21ksoftirqd/023:29:100
766499214,6cyclictest18004-1kworker/u9:121:29:280
766499213,4cyclictest11781-21cpuspeed_turbos00:35:140
766499213,4cyclictest0-21swapper/021:35:180
7664992113,6cyclictest9506-21seq19:14:040
7664992113,6cyclictest15028-21ntpq23:00:220
7664992111,8cyclictest23433-21ssh22:42:350
217292210,2sleep20-21swapper/200:18:112
766799209,2cyclictest141rcu_preempt19:46:243
766699207,9cyclictest26920-21diskmemload23:35:242
766699207,10cyclictest12780-21open_files00:35:212
766699204,14cyclictest9527-21cat23:20:222
766699202,13cyclictest11796-21ssh22:10:132
7666992014,4cyclictest21979-21sh22:41:082
7666992014,4cyclictest20987-21acpi19:40:122
7666992012,6cyclictest29561-21find20:00:102
7666992012,6cyclictest2363-21tune2fs21:15:152
7666992011,6cyclictest23926-21sed19:45:192
7666992010,7cyclictest19256-21cut20:50:182
7666992010,6cyclictest52750irq/139-enp3s0-21:30:142
766599209,7cyclictest53050irq/142-enp3s0-22:40:131
766599207,11cyclictest30613-21timerwakeupswit21:35:261
766599202,13cyclictest18597-21apt-get20:50:121
7665992014,4cyclictest11914-21basename20:35:001
7665992013,5cyclictest53050irq/142-enp3s0-00:15:211
7665992013,4cyclictest22905-21cpuspeed_turbos23:55:131
7665992012,6cyclictest8655-21seq23:44:341
7665992012,6cyclictest6855-21cut22:55:001
7665992012,6cyclictest17629-21seq21:25:411
7665992012,5cyclictest25999-21seq00:20:461
7665992012,5cyclictest25999-21seq00:20:461
7665992011,7cyclictest958-21seq20:07:201
7665992011,7cyclictest4449-21fw_packets00:05:151
7665992011,7cyclictest10925-21seq23:21:111
7665992011,6cyclictest29466-21seq22:46:551
7665992010,7cyclictest560-21dbus-daemon21:45:001
766499209,2cyclictest12-21ksoftirqd/023:56:220
766499208,3cyclictest12-21ksoftirqd/022:25:170
766499202,13cyclictest14319-21apt-get20:40:110
7664992016,2cyclictest121ksoftirqd/019:30:010
766499201,3cyclictest141rcu_preempt21:50:130
7664992012,6cyclictest8305-21seq22:07:110
7664992012,6cyclictest21829-21ls19:40:200
766499201,10cyclictest141rcu_preempt22:20:170
766799198,3cyclictest141rcu_preempt20:50:203
766799198,3cyclictest141rcu_preempt20:25:253
766799196,3cyclictest141rcu_preempt21:00:123
766799195,5cyclictest34-21ksoftirqd/319:35:043
7667991911,6cyclictest25802-21sed19:50:173
7667991910,6cyclictest15640-21grep19:25:283
7666991913,4cyclictest6526-21sh23:18:302
7666991913,4cyclictest32316-21sh00:01:192
7666991913,4cyclictest30610-21sh21:35:262
7666991913,4cyclictest17337-21sh22:38:132
7666991912,5cyclictest5991-21cat20:20:142
7666991912,5cyclictest278-21systemd-journal23:34:222
7666991912,5cyclictest1550-21ntp_kernel_pll_22:50:202
7666991912,4cyclictest636-21runrttasks22:46:272
7666991912,4cyclictest20914-21sh23:53:292
7666991911,6cyclictest4112-21cat00:05:122
7666991911,6cyclictest32303-21basename20:05:182
7666991911,6cyclictest16029-21cron19:30:002
7666991911,6cyclictest10948-21open_files20:30:202
7666991911,5cyclictest8060-21sh22:06:562
7666991910,7cyclictest18220-21cut21:50:222
7666991910,6cyclictest3973-21fw_packets21:40:182
7666991910,6cyclictest11071-21cron00:35:002
766599199,6cyclictest53050irq/142-enp3s0-21:55:141
766599198,3cyclictest141rcu_preempt00:30:171
766599191,7cyclictest14106-21grep22:35:231
7665991913,4cyclictest31267-21kernelversion22:00:181
7665991913,4cyclictest31267-21kernelversion22:00:181
7665991912,5cyclictest29859-21seq23:11:031
7665991912,5cyclictest18022-21seq19:30:291
7665991912,5cyclictest10972-21seq22:57:271
7665991911,6cyclictest4182-21if_enp3s020:15:181
7665991911,6cyclictest15240-21gzip23:00:241
7665991911,6cyclictest10386-21date23:45:201
766599191,15cyclictest25032-21apt-get21:05:131
7665991910,6cyclictest29271-21cat23:35:191
766499192,12cyclictest29575-21apt-get20:00:120
766499191,3cyclictest141rcu_preempt20:30:100
7664991913,4cyclictest23985-21sendmail_mailst21:00:230
7664991913,3cyclictest8976-21sendmail_mailst20:25:220
7664991912,5cyclictest6564-21needreboot20:20:190
7664991912,5cyclictest20528-21date20:55:010
7664991911,6cyclictest27733-21expr23:10:130
7664991911,6cyclictest10900-21cat00:10:150
7664991910,7cyclictest560-21dbus-daemon20:15:000
7664991910,7cyclictest12953-21sed20:35:200
10832190,2chrt0-21swapper/122:25:251
766799187,3cyclictest141rcu_preempt20:10:013
766799186,5cyclictest34-21ksoftirqd/320:35:113
766799183,2cyclictest141rcu_preempt19:40:183
766799182,11cyclictest16500-21apt-get19:30:133
766799182,11cyclictest10020-21apt-get20:30:113
7667991812,5cyclictest34-21ksoftirqd/321:10:003
766699189,7cyclictest21264-21sed20:55:172
766699188,7cyclictest3398-21gltestperf23:40:162
766699188,6cyclictest52750irq/139-enp3s0-00:10:122
766699182,11cyclictest27506-21apt-get23:10:142
7666991813,3cyclictest8365-21sh22:30:522
7666991811,5cyclictest25455-21ls19:50:142
7666991811,5cyclictest25285-21expr21:05:142
7666991811,5cyclictest19183-21entropy19:35:152
7666991811,5cyclictest12429-21cat20:35:152
7666991810,6cyclictest7954-21cat20:25:132
7666991810,6cyclictest30642-21expr22:00:132
7666991810,6cyclictest30642-21expr22:00:132
7666991810,6cyclictest16394-21cat20:45:012
7666991810,5cyclictest17091-21idleruntime19:30:172
766599189,7cyclictest23556-21cut19:45:161
766599187,4cyclictest23797-21ssh22:19:021
766599182,12cyclictest20851-21apt-get23:05:141
7665991813,3cyclictest13052-21sh21:46:231
7665991813,3cyclictest12386-21sh00:10:291
7665991812,4cyclictest26920-21diskmemload22:30:051
7665991812,4cyclictest17200-21sh22:14:101
7665991811,5cyclictest27609-21expr19:55:131
7665991811,5cyclictest19013-21cat19:35:141
7665991811,5cyclictest12165-21seq20:35:131
7665991810,6cyclictest9563-21seq19:14:481
7665991810,6cyclictest28870-21cat21:35:011
7665991810,6cyclictest27489-21seq21:10:081
7664991812,5cyclictest12-21ksoftirqd/020:00:000
7664991812,4cyclictest5460-21latency_hist20:20:000
7664991812,4cyclictest25230-21expr21:05:140
7664991812,3cyclictest32626-21ntp_offset20:05:210
7664991811,5cyclictest20932-21cat20:55:140
766499181,14cyclictest4982-21ntp_states00:05:200
7664991810,6cyclictest19317-21grep19:35:170
766799172,10cyclictest5736-21apt-get20:20:133
7667991712,3cyclictest27391-21/usr/sbin/munin19:55:233
7666991712,3cyclictest636-21runrttasks20:47:452
7666991712,3cyclictest15021-21sh23:25:062
766599179,6cyclictest3135-21seq20:12:461
766599179,6cyclictest26699-21awk19:50:271
766599179,6cyclictest14668-21sed19:25:181
766599172,10cyclictest16459-21apt-get20:45:131
7665991710,5cyclictest6178-21grep20:20:151
7665991710,5cyclictest22420-21sed19:40:261
7664991712,3cyclictest17173-21latency20:45:170
7664991710,5cyclictest23342-21expr19:45:140
766699169,5cyclictest27697-21expr19:55:132
7666991611,3cyclictest8091-21fschecks_count19:10:152
7666991611,3cyclictest1603-21cstates20:10:142
766599168,6cyclictest15913-21seq20:41:231
766599159,4cyclictest8358-21head20:25:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional