You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-30 - 18:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Tue Jun 30, 2026 12:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15080217497,25sleep00-21swapper/007:09:130
2224121630,2sleep00-21swapper/008:40:140
261021430,2sleep00-21swapper/007:55:030
149722143106,26sleep30-21swapper/307:07:523
150872139104,24sleep20-21swapper/207:09:192
149032139103,25sleep10-21swapper/107:06:591
1147221110,2sleep10-21swapper/112:30:081
80321000,2sleep10-21swapper/111:33:301
76212640,9sleep228-21ksoftirqd/209:13:472
25332640,2sleep00-21swapper/011:35:010
226732640,1sleep00-21swapper/012:38:140
14442610,1sleep10-21swapper/109:56:471
225822600,2sleep00-21swapper/009:49:080
30342570,4sleep1221ksoftirqd/109:10:151
135012560,3sleep213505-21cpuspeed_turbos08:20:132
323282550,1sleep30-21swapper/310:44:523
20032550,1sleep00-21swapper/010:45:250
170512540,2sleep10-21swapper/110:32:541
170512540,2sleep10-21swapper/110:32:541
98022520,3chrt0-21swapper/210:51:222
295932520,2sleep30-21swapper/309:30:123
203782500,2sleep10-21swapper/107:20:171
15412995012,28cyclictest13071-21sh12:06:040
1541299500,40cyclictest30019-21seq07:41:150
166492480,1sleep10-21swapper/109:44:551
1541299488,23cyclictest29272-21sh09:30:030
279422470,1sleep30-21swapper/311:29:553
142672470,1sleep10-21swapper/112:07:181
223752460,1sleep20-21swapper/207:25:152
1541299468,25cyclictest20688-21seq11:48:220
1541299447,21cyclictest2154-21diskmemload09:16:410
15412994411,20cyclictest23112-21sh12:14:330
210162430,1sleep10-21swapper/110:35:241
1541299411,5cyclictest5466-21sh11:12:340
144652400,1sleep20-21swapper/209:42:442
15412993914,23cyclictest27998-21seq07:38:010
1541299381,6cyclictest26392-21sh11:52:110
15412993613,20cyclictest32250-21df_inode10:20:130
15412993613,20cyclictest2063-21sh11:58:260
15412993612,21cyclictest25380-21seq11:27:190
15412993610,24cyclictest13245-21cut08:20:000
1541299360,5cyclictest32291-21ssh11:32:340
266822350,9sleep122-21ksoftirqd/112:16:251
15412993512,21cyclictest22839-21seq11:01:020
15412993511,22cyclictest25914-21seq08:49:120
15412993510,22cyclictest31735-21cat09:00:240
1541299350,5cyclictest31699-21ssh10:44:140
15414993414,5cyclictest16040-21/usr/sbin/munin11:45:192
1541299349,22cyclictest21644-21seq08:39:330
1541299348,23cyclictest32690-21/usr/sbin/munin07:50:230
1541299343,27cyclictest75-21kswapd012:25:110
15412993413,18cyclictest22816-21sh07:25:200
15412993410,21cyclictest13275-21sh11:42:360
1541299340,8cyclictest29257-21ssh09:53:530
15414993313,8cyclictest23077-21awk11:25:242
1541299339,21cyclictest25333-21expr10:15:150
1541299339,21cyclictest12963-21expr10:30:140
1541299339,21cyclictest12963-21expr10:30:130
1541299331,6cyclictest6140-21/usr/sbin/munin10:25:220
1541299331,4cyclictest32612-21latency_hist07:50:000
15412993314,16cyclictest10455-21unixbench-2d08:10:230
1541299331,29cyclictest8201-21ssh10:50:210
15412993311,19cyclictest781-21sh12:21:150
15412993311,19cyclictest6735-21sh09:36:430
15412993311,19cyclictest5066-21sh10:00:110
15412993310,20cyclictest23773-21sh10:37:580
1541299330,7cyclictest16374-21df09:20:130
1541299329,20cyclictest8548-21seq08:06:340
1541299328,21cyclictest18419-21sh10:10:090
1541299326,2cyclictest0-21swapper/012:00:200
1541299325,24cyclictest0-21swapper/010:55:100
1541299321,9cyclictest2261-21ls09:10:000
15412993210,19cyclictest28762-21sh12:18:300
15412993210,19cyclictest15876-21sh09:44:070
1541299320,5cyclictest9562-21mailstats11:15:220
1541299320,5cyclictest9562-21mailstats11:15:210
1541299320,4cyclictest30548-21ssh11:06:530
1541299320,3cyclictest28940-21hddtemp_smartct08:55:170
1541299319,19cyclictest19586-21cut07:20:000
1541299318,20cyclictest14307-21sh12:31:300
1541299317,5cyclictest6510-21seq08:02:540
1541299317,20cyclictest17816-21cut08:30:130
1541299316,5cyclictest26720-21grep08:50:170
1541299312,26cyclictest17030-21ssh10:09:000
1541299311,9cyclictest24853-21ssh09:25:490
15414993011,6cyclictest28-21ksoftirqd/210:05:142
1541299307,5cyclictest4052-21timerandwakeup09:10:260
1541299306,6cyclictest25646-21seq07:31:350
1541299301,8cyclictest20648-21ntpq07:20:210
1541299301,8cyclictest15119-21ssh11:20:110
15414992911,5cyclictest28-21ksoftirqd/212:05:182
1541299291,7cyclictest16159-21/usr/sbin/munin08:25:160
1541299290,9cyclictest32285-21grep09:55:260
45982280,3sleep14605-21sh12:00:151
236772280,2sleep30-21swapper/311:01:543
1541399284,10cyclictest16088-21ssh11:45:121
1541299281,7cyclictest13447-21cat08:20:140
1541299281,6cyclictest16101-21irqstats07:10:170
15414992717,8cyclictest12142-21ntp_states08:15:212
15414992710,5cyclictest52950irq/141-enp3s0-11:20:122
94792260,1sleep20-21swapper/212:28:212
183162260,2sleep10-21swapper/111:21:541
1541399261,1cyclictest0-21swapper/107:11:391
15413992611,12cyclictest14743-21gpgv11:20:011
15413992611,12cyclictest14743-21gpgv11:20:001
134552260,1chrt0-21swapper/211:42:462
211882250,2sleep20-21swapper/210:35:242
15415992515,7cyclictest5245-21df12:25:143
1541599251,23cyclictest0-21swapper/307:11:403
1541499251,23cyclictest0-21swapper/207:11:202
136322240,2sleep30-21swapper/309:18:043
1541399238,10cyclictest27218-21ssh11:05:041
1541399233,15cyclictest29632-21ssh11:55:041
15413992314,6cyclictest14463-21grep08:20:221
44112220,1sleep10-21swapper/110:47:401
286612220,2sleep10-21swapper/109:53:171
223512220,9sleep334-21ksoftirqd/312:13:463
218622220,1sleep30-21swapper/311:49:333
1541599228,3cyclictest141rcu_preempt12:32:543
1541599228,12cyclictest141rcu_preempt10:30:443
1541599228,12cyclictest141rcu_preempt10:30:433
1541599221,12cyclictest141rcu_preempt09:10:043
15413992213,7cyclictest3293-21irqstats07:55:181
15413992213,7cyclictest28192-21date11:30:001
1541599211,5cyclictest0-21swapper/310:49:533
15415992113,6cyclictest24347-21diskstats12:15:123
15415992112,6cyclictest10692-21run-parts11:40:243
15415992111,8cyclictest11489-21ssh11:16:553
15415992111,8cyclictest11489-21ssh11:16:543
15415992111,2cyclictest141rcu_preempt10:57:013
1541599210,7cyclictest1446-21ssh10:20:393
1541499218,11cyclictest9000-1kworker/u9:210:55:142
15414992113,6cyclictest3885-21find07:55:232
15414992113,6cyclictest14425-21ls10:30:262
15414992113,6cyclictest14425-21ls10:30:252
15414992112,7cyclictest19289-21seq08:31:472
15414992111,7cyclictest31168-21tune2fs12:20:152
1541399219,7cyclictest305822chrt07:45:141
15413992111,7cyclictest28993-21hddtemp_smartct07:40:161
1541399211,14cyclictest6215-21ssh12:25:241
282872200,2chrt0-21swapper/110:40:451
1541599209,8cyclictest12252-21tune2fs10:05:153
1541599208,4cyclictest8299-21sh10:01:563
1541599206,12cyclictest141rcu_preempt11:07:403
1541599202,4cyclictest0-21swapper/311:11:093
1541599202,13cyclictest10036-21kworker/u8:011:20:123
15415992012,6cyclictest24786-21ls07:30:183
15415992011,6cyclictest29508-21which08:55:243
15415992010,9cyclictest15410-21cyclictest10:27:063
15415992010,8cyclictest28679-21ssh09:53:183
1541499209,6cyclictest11556-21ssh12:30:122
1541499207,10cyclictest25729-1kworker/u9:110:45:132
1541499202,2cyclictest141rcu_preempt12:10:122
1541499202,13cyclictest16040-21ssh09:20:132
15414992012,6cyclictest611-21df07:50:152
15414992012,6cyclictest31402-21ls07:45:192
15414992012,5cyclictest28275-21cat11:05:182
15414992012,5cyclictest20658-21ntp_kernel_pll_07:20:202
15414992012,5cyclictest17752-21tail09:45:152
15414992011,6cyclictest29945-21fw_packets09:30:152
1541499201,13cyclictest9082-21chrt08:10:102
15414992010,8cyclictest11370-21latency_hist10:05:012
1541399208,9cyclictest0-21swapper/111:10:121
1541399203,11cyclictest22317-21apt-get11:50:021
15413992012,6cyclictest23455-21seq12:39:031
15413992012,5cyclictest23248-21ls08:40:241
15413992011,5cyclictest52850irq/140-enp3s0-10:20:111
15413992010,7cyclictest25090-21grep08:45:221
1541399200,8cyclictest229842rm11:01:111
99912190,10sleep122-21ksoftirqd/109:15:181
1541599199,7cyclictest26232-21ls07:35:123
1541599192,12cyclictest4051-21apt-get12:00:123
1541599191,4cyclictest141rcu_preempt09:24:553
1541599191,4cyclictest0-21swapper/310:50:583
15415991913,4cyclictest1381-21sh11:57:443
15415991912,5cyclictest23200-21expr11:50:133
15415991912,5cyclictest19868-21cat10:10:253
15415991912,5cyclictest11748-21cat12:05:193
15415991911,6cyclictest32650-21seq09:05:093
15415991911,6cyclictest20618-21awk10:35:203
1541499199,7cyclictest29261-21date07:40:192
1541499198,7cyclictest52950irq/141-enp3s0-09:35:122
1541499197,3cyclictest141rcu_preempt12:15:112
15414991913,4cyclictest28505-21sh10:17:022
15414991912,5cyclictest26284-21expr07:35:132
15414991912,5cyclictest21018-21tune2fs11:00:172
15414991912,5cyclictest18724-21which07:15:232
15414991912,4cyclictest2154-21diskmemload12:00:252
15414991911,6cyclictest19882-21awk12:35:262
15414991910,7cyclictest30373-21cron09:55:012
15414991910,6cyclictest24054-21mailstats09:25:222
15414991910,5cyclictest0-21swapper/211:35:132
1541399199,6cyclictest52850irq/140-enp3s0-10:25:121
1541399198,7cyclictest52850irq/140-enp3s0-10:10:131
1541399191,6cyclictest0-21swapper/109:21:141
15413991913,3cyclictest7068-21sh10:50:071
15413991911,6cyclictest4393-21cat09:35:171
15413991911,6cyclictest278-21systemd-journal08:55:001
15413991911,6cyclictest16065-21seq10:55:581
1541399191,15cyclictest17015-21ssh09:45:031
1541599189,6cyclictest15637-21grep08:25:123
1541599184,11cyclictest28603-21cpuspeed_turbos11:30:153
1541599182,12cyclictest18192-21taskset12:35:123
1541599182,11cyclictest4804-21apt-get08:00:123
1541599182,11cyclictest3726-21apt-get09:35:143
1541599181,4cyclictest0-21swapper/311:35:023
15415991813,3cyclictest25725-21sh10:15:183
15415991812,4cyclictest34-21ksoftirqd/308:35:003
15415991812,4cyclictest30586-21sh12:20:073
15415991812,4cyclictest24540-21sh09:25:313
15415991812,3cyclictest21885-21acpi07:25:113
15415991811,5cyclictest30977-21expr09:55:153
15415991810,6cyclictest3081-21perl07:55:143
15415991810,6cyclictest30728-21grep07:45:133
15415991810,6cyclictest20377-21if_enp3s007:20:173
15415991810,6cyclictest11399-21date09:40:193
15415991810,5cyclictest26796-21seq08:50:163
15415991810,5cyclictest22772-21cat08:40:183
1541499189,7cyclictest21683-21cron08:39:592
1541499189,6cyclictest6431-21expr10:25:142
1541499189,6cyclictest30271-21fschecks_time11:55:172
1541499188,8cyclictest16327-1kworker/u9:210:20:132
1541499182,11cyclictest22946-21apt-get11:50:122
1541499181,6cyclictest9292-21apt-get09:15:112
15414991813,3cyclictest590-21sh11:33:182
15414991813,3cyclictest30969-21tr09:55:142
15414991812,4cyclictest52950irq/141-enp3s0-10:10:092
15414991812,3cyclictest10683-21sh11:16:062
15414991812,3cyclictest10683-21sh11:16:052
15414991810,6cyclictest32421-21cron09:05:012
15414991810,6cyclictest26568-21df08:50:152
15414991810,6cyclictest24698-21cat08:45:162
15414991810,6cyclictest2315-21basename11:10:182
15414991810,5cyclictest26205-21/usr/sbin/munin10:40:112
1541399189,7cyclictest560-21dbus-daemon07:30:011
1541399187,9cyclictest23596-21missed_timers09:25:191
1541399183,12cyclictest17346-21apt-get12:10:111
1541399181,7cyclictest30926-21seq10:19:291
15413991811,5cyclictest13070-21tr10:05:211
1541399181,15cyclictest10958-21cron08:15:001
15413991810,6cyclictest18270-21cat08:30:161
1541599179,6cyclictest12026-21cat08:15:193
1541599179,5cyclictest25223-21mailstats08:45:223
1541599171,7cyclictest30513-21/usr/sbin/munin09:00:123
15415991712,3cyclictest19309-21sh09:45:503
15415991712,3cyclictest17574-21/usr/sbin/munin07:15:233
15415991712,3cyclictest13383-21apt08:20:113
15415991711,4cyclictest7821-21netstat08:05:203
1541599171,11cyclictest28441-21apt-get07:40:133
15415991710,5cyclictest1199-21ntpq07:50:203
1541499179,5cyclictest28-21ksoftirqd/209:05:152
15414991710,5cyclictest28-21ksoftirqd/208:25:152
15414991710,5cyclictest28-21ksoftirqd/208:05:122
15414991710,4cyclictest29779-21unixbench-2d08:55:272
1541399179,6cyclictest28678-21cat08:55:151
1541399179,6cyclictest26355-21expr07:35:141
1541399179,6cyclictest17314-21seq08:28:541
1541399179,5cyclictest7409-21gltestperf08:05:171
1541399177,7cyclictest560-21dbus-daemon09:10:011
15413991712,3cyclictest11196-21/usr/sbin/munin08:15:271
1541399171,14cyclictest5650-21ssh10:00:141
1541399171,14cyclictest10361-21ssh11:40:201
1541399171,11cyclictest29431-21ssh09:30:111
15413991710,5cyclictest32625-21date09:05:001
15413991710,5cyclictest17929-21df07:15:151
1541599168,6cyclictest20352-21sed08:35:163
1541599162,10cyclictest9079-21apt-get08:10:103
1541499168,6cyclictest23844-21cron08:45:002
1541399168,6cyclictest20557-21latency08:35:191
15413991612,3cyclictest309852sleep112:20:141
15413991611,3cyclictest828-21cut07:50:161
15413991611,3cyclictest6142-21timerwakeupswit08:00:241
15413991611,3cyclictest24013-21/usr/sbin/munin07:30:151
15413991610,4cyclictest2154-21diskmemload11:35:081
15414991510,3cyclictest4974-21cstates08:00:122
15414991510,3cyclictest24209-21cstates07:30:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional