You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-21 - 09:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Dec 21, 2025 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
61809521120,3sleep20-21swapper/221:55:302
6611082980,5sleep00-21swapper/000:00:160
5733962970,5sleep20-21swapper/219:50:222
197399820,11rtkit-daemon4497-21PK-Backend19:26:093
197399820,11rtkit-daemon4497-21PK-Backend19:26:093
197399810,68rtkit-daemon0-21swapper/219:05:192
55662227848,20sleep10-21swapper/119:05:071
55827599660,61cyclictest1972-21rtkit-daemon22:06:200
55805426444,14sleep00-21swapper/019:08:520
55777626439,20sleep30-21swapper/319:05:503
5592952620,13sleep0131ksoftirqd/019:10:280
6437312570,17sleep234-21ksoftirqd/223:10:192
6437312570,17sleep234-21ksoftirqd/223:10:192
558279994837,8cyclictest26-21ksoftirqd/122:46:091
5697672470,8sleep14497-21PK-Backend19:40:191
558285994725,18cyclictest1054-21snmpd20:56:213
558279994724,19cyclictest605323-21hddtemp_smartct21:20:191
55827999471,42cyclictest1972-21rtkit-daemon21:03:171
558281994623,18cyclictest574574-21/usr/sbin/munin19:55:322
55827999459,18cyclictest26-21ksoftirqd/123:15:001
55827999459,18cyclictest26-21ksoftirqd/123:15:001
55827999458,20cyclictest26-21ksoftirqd/119:35:001
6145642440,5sleep00-21swapper/021:45:300
558285994422,18cyclictest1054-21snmpd20:41:183
558285994422,18cyclictest1054-21snmpd00:06:593
558279994420,8cyclictest26-21ksoftirqd/120:10:531
558279994413,14cyclictest26-21ksoftirqd/122:55:001
558279994410,19cyclictest26-21ksoftirqd/122:05:221
558285994325,13cyclictest438-20systemd-journal00:15:003
558285994323,14cyclictest42-21ksoftirqd/319:44:233
55828599431,19cyclictest6249982ntpq22:15:243
558279994314,15cyclictest26-21ksoftirqd/121:52:121
55827999431,22cyclictest438-20systemd-journal21:50:001
197399430,36rtkit-daemon2221-21packagekitd00:33:091
197399430,34rtkit-daemon558269-21cyclictest22:04:281
197399430,34rtkit-daemon558269-21cyclictest22:04:281
6198452420,7chrt4497-21PK-Backend22:00:292
6198452420,7chrt4497-21PK-Backend22:00:282
558285994223,6cyclictest42-21ksoftirqd/320:10:023
558279994211,7cyclictest26-21ksoftirqd/100:14:531
197399420,35rtkit-daemon4497-21PK-Backend22:12:261
197399420,33rtkit-daemon558269-21cyclictest23:03:451
558285994127,9cyclictest626165-21tr22:20:143
558285994126,12cyclictest42-21ksoftirqd/323:11:543
558285994126,12cyclictest42-21ksoftirqd/323:11:543
558281994120,17cyclictest581437-21users20:10:332
55827999418,18cyclictest26-21ksoftirqd/122:30:011
55827999414,8cyclictest26-21ksoftirqd/120:30:011
558279994125,12cyclictest597719-21cron21:00:001
558279994125,11cyclictest438-20systemd-journal19:16:091
55827999411,5cyclictest4497-21PK-Backend21:30:011
55827999411,23cyclictest26-21ksoftirqd/122:25:011
5918912400,5sleep00-21swapper/020:40:260
55828599406,18cyclictest42-21ksoftirqd/319:35:133
558285994023,5cyclictest42-21ksoftirqd/323:30:013
558285994023,13cyclictest42-21ksoftirqd/321:10:013
558285994023,13cyclictest42-21ksoftirqd/320:20:243
558285994022,6cyclictest42-21ksoftirqd/323:10:013
558285994022,6cyclictest42-21ksoftirqd/321:40:003
558285994022,5cyclictest42-21ksoftirqd/300:30:153
558285994019,17cyclictest42-21ksoftirqd/321:25:003
558285994019,17cyclictest1054-21snmpd20:26:153
55827999407,18cyclictest26-21ksoftirqd/120:10:001
55827999405,18cyclictest26-21ksoftirqd/100:30:011
558279994014,8cyclictest26-21ksoftirqd/100:00:221
558275994023,12cyclictest438-20systemd-journal21:35:000
558275994022,14cyclictest624629-21df22:15:150
6599792390,7chrt4497-21PK-Backend23:55:261
6387022390,7chrt4497-21PK-Backend22:55:233
55828599392,32cyclictest573950-21users19:50:313
558285993923,12cyclictest1033-21gdbus20:32:073
558285993922,6cyclictest42-21ksoftirqd/300:15:293
558285993922,6cyclictest42-21ksoftirqd/300:15:283
558285993922,13cyclictest647177-21df23:20:173
558285993922,13cyclictest42-21ksoftirqd/323:30:293
558285993921,6cyclictest42-21ksoftirqd/321:20:003
558285993921,14cyclictest42-21ksoftirqd/322:00:293
558285993921,14cyclictest42-21ksoftirqd/322:00:293
558281993916,11cyclictest570765-21systemd-udevd19:44:242
55827999391,22cyclictest565422-21seq19:27:341
55827999391,22cyclictest565422-21seq19:27:341
55827999391,21cyclictest669664-21turbostat00:25:011
55827999391,20cyclictest2746-21gmain20:36:451
558275993923,12cyclictest1-21systemd22:25:000
5599642380,6sleep1251rcuc/119:15:001
55828599389,25cyclictest652789-21/usr/sbin/munin23:35:233
55828599382,7cyclictest575681-21users19:55:313
558285993827,7cyclictest674859-21sh00:39:593
558285993821,6cyclictest42-21ksoftirqd/300:25:273
558285993821,5cyclictest42-21ksoftirqd/320:00:243
558285993821,5cyclictest42-21ksoftirqd/320:00:243
558285993820,14cyclictest1054-21snmpd00:01:583
558285993818,16cyclictest42-21ksoftirqd/321:11:163
558281993827,6cyclictest34-21ksoftirqd/223:00:152
558281993824,10cyclictest34-21ksoftirqd/200:35:242
55828199382,31cyclictest627731-21apt22:25:122
558281993823,11cyclictest635875-21sshd22:47:092
558281993822,12cyclictest923-21dbus-daemon20:10:002
558281993822,11cyclictest438-20systemd-journal19:45:092
558281993821,13cyclictest600181-21cat21:05:192
558281993821,12cyclictest438-20systemd-journal22:41:102
558281993820,13cyclictest595264-21ntpq20:50:232
55827999384,17cyclictest26-21ksoftirqd/121:40:011
55827999384,12cyclictest26-21ksoftirqd/121:05:221
558279993828,6cyclictest26-21ksoftirqd/121:15:121
55827999382,20cyclictest26-21ksoftirqd/121:10:201
558279993815,8cyclictest26-21ksoftirqd/123:21:561
558279993813,5cyclictest141rcu_preempt19:54:341
558279993812,5cyclictest141rcu_preempt22:33:271
558279993811,8cyclictest26-21ksoftirqd/122:16:011
558275993823,11cyclictest599475-21munin-run21:05:000
558275993822,12cyclictest650533-21apt23:30:120
55827599381,9cyclictest663749-21sed00:05:320
558285993723,11cyclictest42-21ksoftirqd/319:45:183
558285993721,5cyclictest42-21ksoftirqd/320:45:273
558285993720,13cyclictest42-21ksoftirqd/321:45:003
558285993719,6cyclictest42-21ksoftirqd/322:30:023
558285993719,5cyclictest42-21ksoftirqd/319:20:013
55828199372,6cyclictest612726-21smartctl21:40:292
558281993722,11cyclictest2397-21gdbus23:59:592
55828199372,13cyclictest2069-21gnome-shell20:02:082
55828199372,13cyclictest2069-21gnome-shell20:02:082
55828199372,11cyclictest924-21NetworkManager00:01:112
55827999376,16cyclictest141rcu_preempt21:45:001
55827999372,19cyclictest592766-21chrt20:45:021
558279993721,12cyclictest642327-21ntpq23:05:251
558279993718,6cyclictest141rcu_preempt22:59:351
558279993715,8cyclictest26-21ksoftirqd/120:30:031
558279993714,9cyclictest26-21ksoftirqd/123:30:121
55827999371,16cyclictest26-21ksoftirqd/122:45:001
55827599372,7cyclictest623655-40lsb_release22:11:040
558275993720,13cyclictest923-21dbus-daemon23:11:550
558275993720,13cyclictest923-21dbus-daemon23:11:540
558275993720,12cyclictest438-20systemd-journal22:45:000
197399370,30rtkit-daemon0-21swapper/021:22:140
558285993626,6cyclictest42-21ksoftirqd/323:19:253
558285993625,6cyclictest42-21ksoftirqd/320:50:183
558285993625,6cyclictest42-21ksoftirqd/320:11:053
558285993620,12cyclictest42-21ksoftirqd/323:55:233
558285993618,6cyclictest42-21ksoftirqd/300:23:043
558285993617,16cyclictest1054-21snmpd21:46:313
558281993621,11cyclictest666286-21cat00:15:132
558281993621,11cyclictest666286-21cat00:15:122
558281993621,11cyclictest592529-21cron20:45:002
558281993620,11cyclictest438-20systemd-journal19:31:092
558281993618,13cyclictest438-20systemd-journal19:30:002
558281993618,13cyclictest438-20systemd-journal19:30:002
558279993625,8cyclictest26-21ksoftirqd/120:50:001
55827999362,13cyclictest26-21ksoftirqd/100:05:141
558279993619,13cyclictest648974-21if_enp2s023:25:201
558279993614,8cyclictest26-21ksoftirqd/122:39:351
558279993612,12cyclictest141rcu_preempt19:20:151
558275993623,9cyclictest608121-21turbostat21:30:000
558275993621,11cyclictest577373-21pluginstate20:00:290
558275993621,11cyclictest577373-21pluginstate20:00:290
558275993619,13cyclictest584468-21cat20:20:240
55827599361,13cyclictest6387012/usr/sbin/munin22:55:230
55828599358,23cyclictest0-21swapper/320:35:093
558285993525,6cyclictest924-21NetworkManager21:32:143
558285993524,6cyclictest42-21ksoftirqd/323:40:223
558285993524,6cyclictest42-21ksoftirqd/322:11:063
55828599352,29cyclictest632685-21unin-run22:40:013
558285993521,10cyclictest42-21ksoftirqd/323:50:263
558285993520,11cyclictest617765-21ntp_kernel_pll_21:55:243
558285993520,11cyclictest566613-21ntp_kernel_err19:30:253
558285993519,5cyclictest42-21ksoftirqd/321:04:083
558285993519,12cyclictest621229-21ntpq22:05:273
558285993518,5cyclictest42-21ksoftirqd/323:00:323
558285993518,5cyclictest42-21ksoftirqd/322:45:303
558285993517,7cyclictest633017-21cpuspeed_turbos22:40:143
558281993524,7cyclictest656503-21ntpq23:45:282
558281993518,12cyclictest596273-21awk20:55:142
558281993517,12cyclictest438-20systemd-journal22:35:092
558281993514,8cyclictest141rcu_preempt21:45:312
55828199351,30cyclictest6694382chrt00:22:462
55827999354,15cyclictest0-21swapper/100:35:251
55827999353,8cyclictest26-21ksoftirqd/123:42:291
558279993521,10cyclictest26-21ksoftirqd/120:25:011
558279993520,11cyclictest653961-21wc23:40:021
558279993518,13cyclictest609402-21cat21:30:311
55827999351,19cyclictest26-21ksoftirqd/120:52:061
55827599352,28cyclictest634656-21apt22:45:140
558275993521,11cyclictest672889-21sshd00:32:090
558275993520,11cyclictest619148-21diskstats22:00:160
558275993520,11cyclictest619148-21diskstats22:00:160
558275993520,11cyclictest617748-21awk21:55:230
558275993519,11cyclictest438-20systemd-journal23:05:000
558275993518,13cyclictest645440-21df23:15:150
558275993518,13cyclictest599704-21seq21:05:090
558275993518,12cyclictest570084-21ntp_kernel_pll_19:40:240
558285993424,6cyclictest42-21ksoftirqd/321:51:323
55828599342,28cyclictest2069-21gnome-shell23:46:083
55828599341,8cyclictest0-21swapper/322:30:473
558285993418,5cyclictest42-21ksoftirqd/321:25:223
558285993417,12cyclictest438-20systemd-journal20:17:003
558281993427,4cyclictest0-21swapper/221:23:552
558281993423,7cyclictest648715-21tr23:25:132
558281993420,10cyclictest1137-21runrttasks19:36:072
558281993419,11cyclictest608870-21awk21:30:212
558281993418,12cyclictest582548-21cat20:15:222
558281993418,12cyclictest1-21systemd22:15:012
558281993417,12cyclictest624367-21sort22:15:142
55827999347,10cyclictest4497-21PK-Backend19:38:221
55827999344,15cyclictest0-21swapper/119:45:251
558279993427,5cyclictest0-21swapper/119:56:001
558279993420,10cyclictest583782-21grep20:20:021
558279993413,7cyclictest141rcu_preempt20:04:561
558279993413,7cyclictest141rcu_preempt20:04:561
55827999341,10cyclictest4497-21PK-Backend23:15:241
558275993427,4cyclictest0-21swapper/000:22:080
558275993424,6cyclictest560540-21ntp_states19:15:040
558275993423,8cyclictest655949-21tr23:45:160
558275993423,8cyclictest567803-21cpuspeed_turbos19:35:150
558275993422,8cyclictest581475-40apt-check20:11:050
55827599342,27cyclictest630929-21turbostat22:35:000
558275993420,10cyclictest562592-21grep19:20:140
558275993419,11cyclictest648893-21expr23:25:150
558275993419,11cyclictest641768-21grep23:05:150
558275993419,11cyclictest597682-21seq20:59:330
558275993417,12cyclictest438-20systemd-journal22:55:000
197399340,27rtkit-daemon0-21swapper/100:18:301
197399340,27rtkit-daemon0-21swapper/100:18:301
558285993323,5cyclictest42-21ksoftirqd/319:20:263
558285993322,6cyclictest42-21ksoftirqd/319:10:323
558281993322,6cyclictest34-21ksoftirqd/200:13:382
558281993321,8cyclictest642055-21interrupts23:05:212
558281993319,10cyclictest593786-21sed20:45:302
558281993319,10cyclictest2069-21gnome-shell21:54:592
558281993318,11cyclictest663796-21awk00:05:322
558281993318,11cyclictest620917-21ls22:05:232
558281993318,11cyclictest599458-21cron21:05:002
558281993318,11cyclictest560511-21sed19:15:042
558281993317,13cyclictest653030-21ntpq23:35:262
558281993317,12cyclictest923-21dbus-daemon20:25:012
558281993317,12cyclictest670486-21sed00:25:242
558281993317,12cyclictest658536-21sed23:50:272
558281993316,13cyclictest636490-21cat22:50:152
55827999333,9cyclictest26-21ksoftirqd/121:55:171
558275993325,5cyclictest4497-21PK-Backend19:32:150
558275993322,7cyclictest604668-21turbostat21:20:000
558275993320,9cyclictest596014-21perf20:55:000
558275993319,10cyclictest653924-21cat23:40:020
558275993319,10cyclictest564749-21load19:25:250
558275993319,10cyclictest564749-21load19:25:250
558275993317,12cyclictest664956-21cut00:10:200
558275993316,13cyclictest923-21dbus-daemon23:25:010
55827599331,4cyclictest0-21swapper/000:16:310
55827599331,4cyclictest0-21swapper/000:16:310
55827599331,29cyclictest0-21swapper/021:43:350
55827599331,28cyclictest4497-21PK-Backend22:39:090
55827599331,13cyclictest590799-21cron20:40:000
558285993219,8cyclictest636749-21df22:50:163
558281993226,4cyclictest0-21swapper/222:22:232
558281993220,8cyclictest651128-21grep23:30:222
558281993218,10cyclictest607744-21seq21:25:342
558281993218,10cyclictest603452-21expr21:15:162
558281993216,12cyclictest645282-21cat23:15:142
558281993216,12cyclictest588030-21cat20:30:212
558281993215,12cyclictest1074-21lldpd22:30:292
55828199321,4cyclictest4497-21PK-Backend20:25:262
55828199321,28cyclictest0-21swapper/219:22:352
55827999321,17cyclictest26-21ksoftirqd/123:51:191
55827999321,10cyclictest4497-21PK-Backend23:45:141
55827599326,11cyclictest13-21ksoftirqd/000:27:320
55827599325,10cyclictest13-21ksoftirqd/019:55:030
55827599322,11cyclictest674860-21apt00:39:590
558275993217,11cyclictest659615-21forks23:55:180
558275993216,12cyclictest657873-21diskstats23:50:160
55827599321,4cyclictest4497-21PK-Backend19:45:420
55827599321,11cyclictest141rcu_preempt20:35:000
558281993119,8cyclictest611739-21wc21:40:012
558281993116,11cyclictest923-21dbus-daemon21:15:012
55828199311,4cyclictest0-21swapper/219:11:382
558281993114,13cyclictest654531-21kernelversion23:40:212
55828199311,28cyclictest0-21swapper/222:58:022
558275993125,4cyclictest0-21swapper/021:39:320
558275993121,7cyclictest615480-21tr21:50:130
558275993115,12cyclictest602177-21ntpq21:10:230
558275993115,11cyclictest438-20systemd-journal19:55:000
55827599311,27cyclictest4497-21PK-Backend20:15:240
558281993016,10cyclictest590137-21proc_pri20:35:272
558281993015,11cyclictest647534-21ntpq23:20:262
558281993012,6cyclictest141rcu_preempt00:33:282
558275993019,8cyclictest585650-21ls20:25:180
558275993019,7cyclictest628344-21missed_timers22:25:230
558275993018,8cyclictest578336-21/usr/sbin/munin20:05:220
558275993016,10cyclictest593833-21grep20:45:300
558275992820,4cyclictest4497-21PK-Backend23:41:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional