You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-09 - 04:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Fri Jan 09, 2026 00:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1973991040,9rtkit-daemon1972-21rtkit-daemon20:44:590
1973991010,9rtkit-daemon1972-21rtkit-daemon19:35:450
1973991010,6rtkit-daemon1972-21rtkit-daemon23:55:410
1973991000,9rtkit-daemon1972-21rtkit-daemon23:33:480
197399980,9rtkit-daemon1972-21rtkit-daemon23:23:450
197399980,8rtkit-daemon1972-21rtkit-daemon21:21:480
197399980,8rtkit-daemon1972-21rtkit-daemon21:21:480
197399970,8rtkit-daemon1972-21rtkit-daemon20:38:530
197399960,9rtkit-daemon1972-21rtkit-daemon19:43:160
197399950,8rtkit-daemon1972-21rtkit-daemon20:12:330
197399910,9rtkit-daemon1972-21rtkit-daemon23:29:240
31854512820,6sleep14497-21PK-Backend22:15:241
311711227846,23sleep20-21swapper/219:05:122
311831027741,13sleep30-21swapper/319:06:533
197399760,9rtkit-daemon1972-21rtkit-daemon20:44:011
31629132710,4sleep10-21swapper/121:10:281
311751026940,22sleep00-21swapper/019:05:170
311708326944,15sleep10-21swapper/119:05:101
32244222680,16chrt34-21ksoftirqd/200:06:532
311875899619,14cyclictest3138466-40apt-check20:01:580
197399600,9rtkit-daemon1972-21rtkit-daemon21:18:290
197399600,9rtkit-daemon1972-21rtkit-daemon21:18:290
197399600,9rtkit-daemon1972-21rtkit-daemon19:58:350
197399600,9rtkit-daemon1972-21rtkit-daemon19:10:450
197399580,9rtkit-daemon1972-21rtkit-daemon20:34:440
197399570,8rtkit-daemon1972-21rtkit-daemon20:45:060
197399570,8rtkit-daemon1972-21rtkit-daemon19:46:020
197399560,8rtkit-daemon1972-21rtkit-daemon23:27:281
31699792550,6sleep0161rcuc/021:30:270
197399550,8rtkit-daemon1972-21rtkit-daemon20:16:140
197399530,9rtkit-daemon1972-21rtkit-daemon19:22:280
197399520,8rtkit-daemon1972-21rtkit-daemon19:15:030
32040402510,6chrt4497-21PK-Backend23:10:012
3118758995125,9cyclictest13-21ksoftirqd/023:10:260
31364712500,7sleep1251rcuc/119:56:521
3118761995027,18cyclictest3202740-21df23:05:151
3118758995010,5cyclictest141rcu_preempt00:22:480
3118761994925,9cyclictest26-21ksoftirqd/119:25:271
32229712480,6chrt411rcuc/300:05:003
311876199487,15cyclictest141rcu_preempt21:45:011
3118761994813,9cyclictest26-21ksoftirqd/120:00:091
311875899489,5cyclictest141rcu_preempt22:40:540
197399480,41rtkit-daemon26-21ksoftirqd/120:37:091
311876899461,9cyclictest0-21swapper/222:40:002
311876199463,18cyclictest26-21ksoftirqd/122:11:521
311875899461,8cyclictest3216009-21turbostat23:45:010
197399460,36rtkit-daemon3129179-21processes19:35:261
3118768994523,14cyclictest34-21ksoftirqd/221:10:542
311876199454,20cyclictest26-21ksoftirqd/120:30:011
311876199454,17cyclictest0-21swapper/122:35:151
311876199452,19cyclictest26-21ksoftirqd/119:15:141
311876199451,20cyclictest26-21ksoftirqd/100:00:161
3118758994528,12cyclictest3200249-21systemd-udevd22:59:000
31336322440,4sleep30-21swapper/319:50:143
3118768994426,6cyclictest34-21ksoftirqd/223:25:182
3118768994425,14cyclictest34-21ksoftirqd/223:15:272
3118768994424,13cyclictest34-21ksoftirqd/220:00:022
3118768994423,17cyclictest34-21ksoftirqd/223:35:132
3118768994418,21cyclictest34-21ksoftirqd/219:25:142
311876199441,17cyclictest3178895-21awk21:55:311
3118758994432,9cyclictest13-21ksoftirqd/021:39:300
3118758994410,5cyclictest141rcu_preempt21:00:000
3118758994410,5cyclictest141rcu_preempt21:00:000
31820112430,5sleep00-21swapper/022:05:250
3118770994322,17cyclictest3161555-21cron21:10:003
3118768994325,14cyclictest34-21ksoftirqd/223:59:592
3118768994325,14cyclictest34-21ksoftirqd/221:40:022
3118768994325,14cyclictest34-21ksoftirqd/219:45:132
3118768994325,14cyclictest34-21ksoftirqd/219:15:022
3118768994325,14cyclictest34-21ksoftirqd/200:30:012
3118768994324,15cyclictest34-21ksoftirqd/221:30:002
3118768994324,14cyclictest34-21ksoftirqd/200:22:532
3118768994322,7cyclictest34-21ksoftirqd/222:59:012
3118768994320,14cyclictest34-21ksoftirqd/220:35:002
311876199435,19cyclictest26-21ksoftirqd/123:45:011
311876199431,19cyclictest26-21ksoftirqd/120:15:131
311876199431,18cyclictest3209056-21runrttasks23:25:001
311876199431,18cyclictest26-21ksoftirqd/123:50:261
311876199431,18cyclictest26-21ksoftirqd/100:11:521
311876199431,18cyclictest26-21ksoftirqd/100:11:521
311876199431,17cyclictest927-21gmain19:47:521
311876199431,17cyclictest26-21ksoftirqd/100:20:131
3118761994310,18cyclictest26-21ksoftirqd/122:30:011
3118758994311,16cyclictest13-21ksoftirqd/022:20:260
3118768994224,14cyclictest34-21ksoftirqd/221:45:022
3118768994224,14cyclictest34-21ksoftirqd/200:35:162
3118768994224,13cyclictest34-21ksoftirqd/220:10:022
3118768994224,13cyclictest34-21ksoftirqd/220:10:022
3118768994223,15cyclictest34-21ksoftirqd/222:20:002
3118768994222,16cyclictest34-21ksoftirqd/223:45:132
3118768994222,16cyclictest34-21ksoftirqd/223:45:132
3118768994221,13cyclictest34-21ksoftirqd/221:20:032
3118768994221,13cyclictest34-21ksoftirqd/221:20:032
3118768994221,13cyclictest34-21ksoftirqd/219:35:132
311876899421,20cyclictest3152002-21cut20:40:262
3118768994211,5cyclictest141rcu_preempt22:30:012
3118761994212,8cyclictest26-21ksoftirqd/121:40:001
311876199421,16cyclictest3134290-21taskset19:50:251
311875899424,5cyclictest141rcu_preempt00:16:120
3118758994224,14cyclictest13-21ksoftirqd/022:52:080
3118758994224,13cyclictest13-21ksoftirqd/023:01:590
3118758994224,13cyclictest13-21ksoftirqd/019:30:200
3118758994222,12cyclictest13-21ksoftirqd/021:25:270
3118768994124,13cyclictest34-21ksoftirqd/221:00:262
3118768994124,12cyclictest34-21ksoftirqd/220:15:002
3118768994123,6cyclictest34-21ksoftirqd/222:10:132
3118768994123,6cyclictest34-21ksoftirqd/219:25:002
3118768994123,6cyclictest34-21ksoftirqd/200:30:232
3118768994123,14cyclictest34-21ksoftirqd/220:35:222
3118768994123,13cyclictest34-21ksoftirqd/220:00:172
3118768994122,14cyclictest34-21ksoftirqd/222:20:302
3118768994120,13cyclictest34-21ksoftirqd/220:15:272
3118768994120,13cyclictest34-21ksoftirqd/219:55:012
311876199414,21cyclictest26-21ksoftirqd/119:30:101
311876199412,20cyclictest26-21ksoftirqd/122:07:141
311876199411,19cyclictest3214258-21cron23:40:001
311876199411,15cyclictest4497-21PK-Backend22:59:021
3118758994131,6cyclictest13-21ksoftirqd/021:08:080
3118758994123,14cyclictest13-21ksoftirqd/000:30:260
3118758994120,13cyclictest13-21ksoftirqd/021:55:210
3118758994115,10cyclictest13-21ksoftirqd/023:19:380
3118758994114,12cyclictest141rcu_preempt22:25:120
3118758994112,5cyclictest141rcu_preempt19:30:000
3118758994111,12cyclictest141rcu_preempt00:00:210
197399410,31rtkit-daemon1972-21rtkit-daemon20:45:241
31395132401,10sleep33139582-21cat20:05:203
31395132401,10sleep33139582-21cat20:05:203
3118770994021,14cyclictest3227044-21df00:15:163
311877099401,26cyclictest141rcu_preempt20:55:213
311877099401,26cyclictest141rcu_preempt20:55:213
311877099401,13cyclictest31518762awk20:40:233
3118768994023,5cyclictest34-21ksoftirqd/200:00:132
3118768994023,13cyclictest34-21ksoftirqd/223:40:132
3118768994023,13cyclictest34-21ksoftirqd/223:35:022
3118768994023,13cyclictest34-21ksoftirqd/221:45:222
3118768994023,13cyclictest34-21ksoftirqd/221:10:022
3118768994023,13cyclictest34-21ksoftirqd/220:50:262
3118768994023,13cyclictest34-21ksoftirqd/219:45:022
3118768994023,13cyclictest34-21ksoftirqd/219:15:052
3118768994023,13cyclictest34-21ksoftirqd/200:10:242
3118768994023,13cyclictest34-21ksoftirqd/200:10:242
3118768994022,14cyclictest34-21ksoftirqd/221:00:002
3118768994022,14cyclictest34-21ksoftirqd/221:00:002
3118768994022,13cyclictest34-21ksoftirqd/220:45:302
3118768994021,15cyclictest34-21ksoftirqd/220:25:242
3118768994021,14cyclictest34-21ksoftirqd/222:30:272
311876199405,15cyclictest0-21swapper/122:35:021
311876199403,20cyclictest26-21ksoftirqd/122:00:131
311876199403,16cyclictest3140971-21cat20:10:131
3118761994016,6cyclictest141rcu_preempt22:45:001
3118761994015,6cyclictest141rcu_preempt19:10:281
311876199401,19cyclictest3196127-21ntp_kernel_err22:45:241
311876199401,19cyclictest26-21ksoftirqd/123:35:011
311876199401,18cyclictest26-21ksoftirqd/120:50:231
311875899409,5cyclictest141rcu_preempt22:01:270
311875899405,8cyclictest13-21ksoftirqd/021:00:130
3118758994028,8cyclictest13-21ksoftirqd/021:10:540
3118758994023,5cyclictest13-21ksoftirqd/023:50:130
3118758994023,5cyclictest13-21ksoftirqd/020:25:200
3118758994023,13cyclictest13-21ksoftirqd/023:35:240
3118758994023,13cyclictest13-21ksoftirqd/022:15:480
3118758994022,5cyclictest13-21ksoftirqd/023:05:130
3118758994022,5cyclictest13-21ksoftirqd/022:10:220
3118758994022,14cyclictest13-21ksoftirqd/022:30:300
3118758994022,14cyclictest13-21ksoftirqd/000:35:160
3118758994016,8cyclictest13-21ksoftirqd/021:40:090
3118770993925,10cyclictest42-21ksoftirqd/322:58:583
3118770993923,11cyclictest438-20systemd-journal22:27:513
3118768993922,5cyclictest34-21ksoftirqd/221:50:312
3118768993922,14cyclictest34-21ksoftirqd/200:20:002
3118768993922,13cyclictest34-21ksoftirqd/220:20:112
3118768993921,14cyclictest34-21ksoftirqd/223:05:012
3118768993921,14cyclictest34-21ksoftirqd/222:50:262
3118768993921,14cyclictest34-21ksoftirqd/221:23:512
3118768993921,14cyclictest34-21ksoftirqd/221:23:512
3118768993921,13cyclictest34-21ksoftirqd/222:00:292
3118768993920,14cyclictest34-21ksoftirqd/223:50:202
3118768993920,14cyclictest34-21ksoftirqd/223:25:002
3118768993920,14cyclictest34-21ksoftirqd/221:30:132
311876199395,19cyclictest26-21ksoftirqd/100:37:251
311876199392,32cyclictest3175832-21munin-run21:50:011
3118761993922,12cyclictest438-20systemd-journal22:53:531
311876199392,10cyclictest26-21ksoftirqd/123:10:261
311876199391,19cyclictest26-21ksoftirqd/100:10:011
311875899399,6cyclictest141rcu_preempt00:05:170
3118758993923,13cyclictest13-21ksoftirqd/022:50:020
3118758993922,5cyclictest13-21ksoftirqd/020:22:520
3118758993922,5cyclictest13-21ksoftirqd/019:50:110
3118758993922,14cyclictest13-21ksoftirqd/020:05:130
3118758993922,14cyclictest13-21ksoftirqd/020:05:130
3118758993921,14cyclictest13-21ksoftirqd/021:50:140
3118758993910,9cyclictest141rcu_preempt00:25:150
311877099385,29cyclictest3166589-21timerandwakeup21:20:293
311877099385,29cyclictest3166589-21timerandwakeup21:20:293
3118770993820,13cyclictest438-20systemd-journal23:05:003
3118768993821,13cyclictest34-21ksoftirqd/223:15:022
3118768993820,5cyclictest34-21ksoftirqd/222:05:292
3118768993820,14cyclictest34-21ksoftirqd/222:40:262
3118768993820,14cyclictest34-21ksoftirqd/221:55:262
311876199384,8cyclictest26-21ksoftirqd/119:20:101
311876199383,18cyclictest26-21ksoftirqd/123:00:141
311876199383,14cyclictest0-21swapper/120:30:111
3118761993828,7cyclictest1137-21runrttasks23:18:491
3118761993827,8cyclictest26-21ksoftirqd/100:00:011
311876199381,16cyclictest438-20systemd-journal21:19:521
311876199381,16cyclictest438-20systemd-journal21:19:521
3118758993822,13cyclictest13-21ksoftirqd/021:45:110
3118758993822,13cyclictest13-21ksoftirqd/020:50:140
3118758993821,13cyclictest13-21ksoftirqd/022:37:530
3118758993820,5cyclictest13-21ksoftirqd/000:10:250
3118758993820,5cyclictest13-21ksoftirqd/000:10:240
311875899381,7cyclictest3217727-21grep23:50:000
311875899381,7cyclictest3217727-21grep23:49:590
311877099372,30cyclictest3213889-21users23:35:313
311877099372,14cyclictest3203127-21ntp_states23:05:263
3118770993721,11cyclictest3218724-21ntp_kernel_pll_23:50:243
3118770993712,10cyclictest3179685-21tr22:00:133
3118770993711,21cyclictest3140738-21/usr/sbin/munin20:10:143
3118768993720,13cyclictest34-21ksoftirqd/222:45:152
3118768993719,6cyclictest34-21ksoftirqd/219:30:122
3118761993722,11cyclictest3168732-21sshd21:28:521
311876199371,15cyclictest3167092-21cron21:25:001
311876199371,15cyclictest3167092-21cron21:25:001
311876199371,14cyclictest0-21swapper/100:30:241
311876199371,11cyclictest3157217-21chrt20:55:271
311876199371,11cyclictest3157217-21chrt20:55:261
311877099369,10cyclictest3191543-21sort22:35:023
3118770993620,12cyclictest3184466-21munin-run22:15:003
311876199368,5cyclictest141rcu_preempt00:25:281
3118761993623,8cyclictest3144349-21sort20:20:121
311876199362,16cyclictest0-21swapper/120:05:131
311876199362,16cyclictest0-21swapper/120:05:131
315069523511,7sleep342-21ksoftirqd/320:37:513
311877099352,28cyclictest3120315-40lsb_release19:11:023
3118770993521,9cyclictest3178542-21ntp_states21:55:233
3118770993520,11cyclictest3195548-21expr22:45:143
3118770993519,11cyclictest3145301-21cat20:20:293
3118770993519,11cyclictest3122147-21ntp_kernel_pll_19:15:243
3118770993518,12cyclictest438-20systemd-journal00:28:183
311877099351,30cyclictest4497-21PK-Backend22:35:343
3118770993512,7cyclictest141rcu_preempt22:40:223
311877099351,13cyclictest3186453-21/usr/sbin/munin22:20:113
311876199355,8cyclictest26-21ksoftirqd/123:50:001
311876199355,8cyclictest26-21ksoftirqd/123:50:001
311876199354,27cyclictest26-21ksoftirqd/121:50:141
3118761993524,8cyclictest26-21ksoftirqd/122:24:081
3118761993515,5cyclictest141rcu_preempt21:00:021
311877099348,22cyclictest42-21ksoftirqd/321:35:503
3118770993423,6cyclictest42-21ksoftirqd/319:20:183
3118770993420,10cyclictest3233710-21cron00:35:013
3118770993419,11cyclictest3158334-21expr21:00:133
3118770993419,11cyclictest2080-21gdbus00:23:083
3118770993418,13cyclictest3156173-21cut20:55:023
3118770993418,12cyclictest3133262-21cron19:50:003
3118770993417,13cyclictest3210062-21ntpq23:25:233
3118770993412,8cyclictest3148140-21if_enp2s020:30:213
311877099341,13cyclictest4497-21PK-Backend21:29:593
3118770993410,5cyclictest141rcu_preempt19:25:183
3118761993417,12cyclictest3130966-21awk19:40:271
3118761993411,6cyclictest141rcu_preempt00:15:271
311876199341,14cyclictest26-21ksoftirqd/121:05:261
32242982330,7chrt4497-21PK-Backend00:05:293
311877099334,22cyclictest0-21swapper/323:10:213
3118770993321,7cyclictest3185203-21irqstats22:15:213
3118770993319,10cyclictest3212620-21ls23:35:023
3118770993317,12cyclictest1054-21snmpd22:53:013
3118770993316,12cyclictest438-20systemd-journal20:20:003
311877099331,29cyclictest4497-21PK-Backend23:47:483
311877099331,29cyclictest4497-21PK-Backend23:47:473
311877099331,29cyclictest4497-21PK-Backend23:19:023
311877099331,29cyclictest4497-21PK-Backend19:56:243
311877099331,29cyclictest4497-21PK-Backend19:36:043
311876199333,15cyclictest0-21swapper/121:30:521
3118770993226,4cyclictest0-21swapper/323:40:563
3118770993222,5cyclictest42-21ksoftirqd/320:46:323
3118770993221,6cyclictest42-21ksoftirqd/321:43:383
3118770993221,6cyclictest42-21ksoftirqd/321:13:243
3118770993219,9cyclictest3164318-21ntpq21:15:233
3118770993219,9cyclictest3164318-21ntpq21:15:233
311877099321,8cyclictest4497-21PK-Backend23:22:383
3118770993217,11cyclictest923-21dbus-daemon21:35:013
3118770993216,13cyclictest4497-21PK-Backend00:40:003
311877099321,10cyclictest3225352-21chrt00:10:203
311877099321,10cyclictest3225352-21chrt00:10:203
311877099313,5cyclictest4497-21PK-Backend00:00:003
311877099312,8cyclictest3175422-21users21:45:303
3118770993120,6cyclictest42-21ksoftirqd/320:01:163
3118770993116,11cyclictest976-21rs:main3
311877099311,27cyclictest4497-21PK-Backend19:40:083
311877099304,10cyclictest42-21ksoftirqd/319:34:483
311877099302,15cyclictest31775402munin-run21:55:003
311877099291,24cyclictest3146082-21tr20:25:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional