You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-10 - 06:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Mon Aug 10, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
205583221320,4sleep30-21swapper/323:25:133
206538421280,10sleep013-21ksoftirqd/023:50:200
201355921260,4sleep33056-21PK-Backend21:25:193
207874221020,3sleep00-21swapper/000:30:130
1390991000,59rtkit-daemon0-21swapper/119:07:591
20010782900,3sleep20-21swapper/220:50:122
196468227539,28sleep30-21swapper/319:09:533
196323027145,16sleep00-21swapper/019:05:120
196459826545,13sleep20-21swapper/219:08:482
19881872580,4sleep33056-21PK-Backend20:15:013
20222082470,4sleep12022212-21ntp_kernel_err21:50:181
20739242440,3sleep126-21ksoftirqd/100:15:181
20460262430,3sleep20-21swapper/222:57:002
20373832430,7sleep02037384-21latency_hist22:35:000
20554162410,3sleep013-21ksoftirqd/023:25:010
1964804993014,5cyclictest26-21ksoftirqd/123:55:111
139099270,4rtkit-daemon1389-21rtkit-daemon23:05:001
139099270,4rtkit-daemon1389-21rtkit-daemon22:03:171
1964811992315,6cyclictest2081713-21sshd00:39:503
1964808992316,5cyclictest1977372-21cron19:44:592
1964808992316,5cyclictest1977372-21cron19:44:592
196480499233,8cyclictest0-21swapper/122:05:121
196480499231,20cyclictest2043823-21ntp_states22:50:181
1964799992314,7cyclictest1981482-21df19:55:150
139099230,19rtkit-daemon1788643-21eadedCompositor22:42:581
20035282220,4sleep33056-21PK-Backend20:55:273
1964811992214,2cyclictest42-21ksoftirqd/321:55:003
196481199220,20cyclictest3056-21PK-Backend22:15:093
196480899220,21cyclictest0-21swapper/219:49:252
196480499229,11cyclictest2030190-21unin-run22:15:001
1964804992220,1cyclictest0-21swapper/100:09:201
1964804992212,7cyclictest436-20systemd-journal20:15:001
196480499220,21cyclictest0-21swapper/120:17:351
196480499220,20cyclictest3056-21PK-Backend00:20:361
1964799992215,5cyclictest13-21ksoftirqd/019:15:140
1964799992214,6cyclictest883-21dbus-daemon23:05:000
1964799992213,7cyclictest2025682-21cat22:00:180
196479999220,21cyclictest1389-21rtkit-daemon00:20:570
19811882210,4sleep1251rcuc/119:55:151
1964811992118,2cyclictest0-21swapper/320:40:123
1964811992112,7cyclictest42-21ksoftirqd/323:45:003
1964811992112,7cyclictest2019113-21awk21:40:233
1964808992119,1cyclictest0-21swapper/223:51:112
196480899210,6cyclictest2036424-21grep22:30:192
196480899210,20cyclictest0-21swapper/222:15:052
196480899210,20cyclictest0-21swapper/221:32:142
196480899210,19cyclictest3056-21PK-Backend23:10:152
196480899210,19cyclictest3056-21PK-Backend00:00:252
196480499218,5cyclictest26-21ksoftirqd/121:25:111
1964804992119,1cyclictest0-21swapper/123:33:241
1964804992119,1cyclictest0-21swapper/122:56:271
1964804992119,1cyclictest0-21swapper/122:47:141
1964804992119,1cyclictest0-21swapper/121:09:131
1964804992119,1cyclictest0-21swapper/121:00:411
1964804992119,1cyclictest0-21swapper/120:26:351
1964804992118,2cyclictest3056-21PK-Backend20:42:101
1964804992115,4cyclictest26-21ksoftirqd/122:35:001
1964804992115,4cyclictest1996979-21sh20:39:591
1964804992115,4cyclictest1996979-21sh20:39:591
196480499210,20cyclictest0-21swapper/119:35:111
196480499210,19cyclictest3056-21PK-Backend21:47:401
1964799992119,1cyclictest0-21swapper/019:50:100
1964799992118,2cyclictest3056-21PK-Backend00:16:090
196479999210,20cyclictest3056-21PK-Backend19:22:590
196479999210,20cyclictest0-21swapper/021:22:380
196481199202,13cyclictest2075081-21runrttasks00:20:103
1964811992018,1cyclictest3056-21PK-Backend22:55:333
1964811992018,1cyclictest3056-21PK-Backend22:26:123
1964811992018,1cyclictest0-21swapper/323:46:563
1964811992018,1cyclictest0-21swapper/323:20:273
1964811992018,1cyclictest0-21swapper/320:02:473
1964811992018,1cyclictest0-21swapper/319:13:143
1964811992014,4cyclictest2019-21gmain23:35:003
1964811992012,6cyclictest436-20systemd-journal19:58:013
1964811992011,7cyclictest2065383-21/usr/sbin/munin23:50:193
1964811992011,6cyclictest1970440-21cron19:25:013
196481199200,1cyclictest3056-21PK-Backend21:31:373
196481199200,19cyclictest3056-21PK-Backend19:51:123
196481199200,19cyclictest3056-21PK-Backend19:39:223
196481199200,19cyclictest3056-21PK-Backend00:11:093
196481199200,19cyclictest0-21swapper/323:13:023
196481199200,19cyclictest0-21swapper/321:56:113
196481199200,19cyclictest0-21swapper/319:34:563
196481199200,19cyclictest0-21swapper/319:29:563
196481199200,19cyclictest0-21swapper/300:00:143
196481199200,18cyclictest3056-21PK-Backend23:01:063
196481199200,18cyclictest3056-21PK-Backend22:48:013
1964808992018,1cyclictest0-21swapper/223:03:542
1964808992018,1cyclictest0-21swapper/222:09:122
1964808992018,1cyclictest0-21swapper/220:41:382
1964808992018,1cyclictest0-21swapper/220:31:022
1964808992018,1cyclictest0-21swapper/220:07:302
1964808992018,1cyclictest0-21swapper/200:05:172
1964808992017,2cyclictest0-21swapper/221:26:122
196480899200,1cyclictest0-21swapper/221:05:042
196480899200,1cyclictest0-21swapper/200:19:432
196480899200,19cyclictest0-21swapper/223:58:142
196480899200,19cyclictest0-21swapper/223:46:522
196480899200,19cyclictest0-21swapper/223:37:542
196480899200,19cyclictest0-21swapper/223:27:322
196480899200,19cyclictest0-21swapper/223:18:122
196480899200,19cyclictest0-21swapper/222:00:252
196480899200,19cyclictest0-21swapper/221:41:402
196480899200,19cyclictest0-21swapper/221:13:082
196480899200,19cyclictest0-21swapper/221:01:252
196480899200,19cyclictest0-21swapper/220:28:122
196480899200,19cyclictest0-21swapper/220:21:362
196480899200,19cyclictest0-21swapper/220:12:232
196480899200,19cyclictest0-21swapper/219:57:582
196480899200,19cyclictest0-21swapper/219:31:152
196480899200,19cyclictest0-21swapper/200:32:342
196480899200,19cyclictest0-21swapper/200:10:102
1964804992018,1cyclictest3056-21PK-Backend19:11:101
1964804992018,1cyclictest0-21swapper/123:17:021
1964804992018,1cyclictest0-21swapper/119:54:031
1964804992015,4cyclictest26-21ksoftirqd/100:40:001
1964804992014,4cyclictest26-21ksoftirqd/120:35:001
196480499200,1cyclictest3056-21PK-Backend22:18:491
196480499200,19cyclictest0-21swapper/123:41:091
196480499200,19cyclictest0-21swapper/121:31:441
196480499200,19cyclictest0-21swapper/120:24:531
196480499200,19cyclictest0-21swapper/120:08:361
196480499200,19cyclictest0-21swapper/100:27:391
196480499200,19cyclictest0-21swapper/100:13:271
196480499200,18cyclictest3056-21PK-Backend23:35:201
1964799992018,1cyclictest0-21swapper/023:58:110
1964799992018,1cyclictest0-21swapper/022:25:170
1964799992018,1cyclictest0-21swapper/022:07:120
1964799992018,1cyclictest0-21swapper/021:43:190
1964799992018,1cyclictest0-21swapper/019:33:540
1964799992018,1cyclictest0-21swapper/000:12:030
1964799992018,1cyclictest0-21swapper/000:04:260
196479999200,1cyclictest0-21swapper/022:50:580
196479999200,19cyclictest0-21swapper/023:42:090
196479999200,19cyclictest0-21swapper/021:35:350
196479999200,19cyclictest0-21swapper/021:13:520
196479999200,19cyclictest0-21swapper/021:00:400
196479999200,19cyclictest0-21swapper/020:32:490
196479999200,19cyclictest0-21swapper/019:40:170
196479999200,19cyclictest0-21swapper/019:40:170
139099200,14rtkit-daemon0-21swapper/019:28:370
139099200,13rtkit-daemon0-21swapper/119:20:111
19890182190,4chrt3056-21PK-Backend20:15:172
1964811991917,1cyclictest3056-21PK-Backend21:10:543
1964811991917,1cyclictest3056-21PK-Backend21:04:203
196481199191,6cyclictest2074086-21ntp_states00:15:203
1964811991912,4cyclictest1095-21runrttasks20:35:203
1964811991912,4cyclictest1095-21runrttasks20:35:193
196481199190,1cyclictest3056-21PK-Backend23:06:403
196481199190,1cyclictest3056-21PK-Backend20:32:293
196481199190,1cyclictest3056-21PK-Backend00:07:033
196481199190,1cyclictest0-21swapper/323:17:423
196481199190,1cyclictest0-21swapper/320:20:253
196481199190,18cyclictest3056-21PK-Backend21:15:553
196481199190,18cyclictest3056-21PK-Backend20:09:113
196481199190,18cyclictest0-21swapper/319:43:153
196481199190,18cyclictest0-21swapper/319:43:143
1964808991917,1cyclictest0-21swapper/222:44:462
1964808991917,1cyclictest0-21swapper/222:38:252
1964808991917,1cyclictest0-21swapper/222:27:292
1964808991917,1cyclictest0-21swapper/220:56:042
1964808991916,2cyclictest0-21swapper/200:23:292
1964808991912,5cyclictest2058239-21cat23:30:242
196480899190,1cyclictest0-21swapper/222:49:122
196480899190,1cyclictest0-21swapper/222:22:432
196480899190,1cyclictest0-21swapper/221:39:242
196480899190,18cyclictest0-21swapper/223:22:522
196480899190,18cyclictest0-21swapper/200:28:022
196480499194,5cyclictest1095-21runrttasks23:53:091
1964804991917,1cyclictest3056-21PK-Backend23:49:411
1964804991917,1cyclictest0-21swapper/123:10:061
1964804991917,1cyclictest0-21swapper/119:46:531
1964804991917,1cyclictest0-21swapper/119:41:541
1964804991917,1cyclictest0-21swapper/119:41:531
1964804991917,1cyclictest0-21swapper/119:16:581
1964804991913,4cyclictest2156-21gnome-shell21:12:091
1964804991912,5cyclictest1973929-21munin-plugin-st19:35:001
1964804991910,6cyclictest2002158-21cron20:55:001
196480499190,1cyclictest0-21swapper/122:39:441
196480499190,1cyclictest0-21swapper/100:00:351
196480499190,17cyclictest3056-21PK-Backend21:43:551
1964799991917,1cyclictest0-21swapper/023:38:030
1964799991917,1cyclictest0-21swapper/022:57:550
1964799991917,1cyclictest0-21swapper/022:44:020
1964799991917,1cyclictest0-21swapper/022:17:390
1964799991917,1cyclictest0-21swapper/021:45:350
1964799991917,1cyclictest0-21swapper/021:30:070
1964799991917,1cyclictest0-21swapper/021:29:000
1964799991917,1cyclictest0-21swapper/021:16:290
1964799991917,1cyclictest0-21swapper/021:05:270
1964799991917,1cyclictest0-21swapper/020:16:260
1964799991917,1cyclictest0-21swapper/020:04:020
1964799991917,1cyclictest0-21swapper/019:45:310
1964799991912,5cyclictest2002551-21cat20:55:160
196479999190,1cyclictest0-21swapper/023:47:360
196479999190,1cyclictest0-21swapper/022:49:030
196479999190,1cyclictest0-21swapper/020:11:330
196479999190,1cyclictest0-21swapper/020:07:140
196479999190,1cyclictest0-21swapper/000:35:490
196479999190,1cyclictest0-21swapper/000:26:090
196479999190,18cyclictest3056-21PK-Backend23:16:280
196479999190,18cyclictest0-21swapper/022:37:060
196479999190,18cyclictest0-21swapper/020:25:060
20435082180,4sleep33056-21PK-Backend22:50:163
1964811991816,1cyclictest0-21swapper/320:19:513
1964811991813,4cyclictest42-21ksoftirqd/321:40:003
1964811991811,5cyclictest2059590-21cat23:35:153
1964811991811,5cyclictest1978363-21awk19:45:193
1964811991810,6cyclictest436-20systemd-journal22:40:003
1964811991810,6cyclictest2020134-21df21:45:143
1964808991816,1cyclictest0-21swapper/221:15:442
1964808991816,1cyclictest0-21swapper/220:39:222
1964808991816,1cyclictest0-21swapper/220:39:212
1964808991813,3cyclictest1965879-21rt-features19:10:192
1964804991816,1cyclictest0-21swapper/121:55:571
1964804991816,1cyclictest0-21swapper/121:21:171
1964804991811,5cyclictest2009495-21sort21:15:151
1964804991810,7cyclictest2078944-21which00:30:161
1964804991810,6cyclictest2033418-21sshd22:22:321
196479999186,2cyclictest141rcu_preempt20:45:110
1964799991816,1cyclictest0-21swapper/023:07:210
1964799991816,1cyclictest0-21swapper/020:40:320
1964799991816,1cyclictest0-21swapper/020:38:500
1964799991816,1cyclictest0-21swapper/020:38:500
1964799991816,1cyclictest0-21swapper/019:38:200
1964799991813,3cyclictest2033631-21idleruntime-cro22:25:000
1964799991810,6cyclictest2054936-21apt-get23:25:000
196481199179,2cyclictest42-21ksoftirqd/300:00:013
196481199178,7cyclictest2039581-21apt-get22:40:033
196481199172,3cyclictest141rcu_preempt22:35:003
1964811991715,1cyclictest3056-21PK-Backend20:45:193
1964811991715,1cyclictest3056-21PK-Backend00:28:133
1964811991714,2cyclictest3056-21PK-Backend20:25:533
196481199171,3cyclictest3056-21PK-Backend21:05:183
196480899172,2cyclictest141rcu_preempt22:55:002
1964808991715,1cyclictest0-21swapper/221:45:042
1964808991715,1cyclictest0-21swapper/220:48:262
1964808991715,1cyclictest0-21swapper/220:03:452
1964808991715,1cyclictest0-21swapper/219:35:402
1964808991715,1cyclictest0-21swapper/200:37:082
1964808991711,5cyclictest2023582-21ls21:55:102
1964808991710,5cyclictest926-21udisksd19:28:042
196480499179,6cyclictest2048446-21expr23:05:131
196480499179,6cyclictest2034631-21ntp_states22:25:201
1964804991715,1cyclictest3056-21PK-Backend21:38:061
1964804991715,1cyclictest0-21swapper/120:00:391
1964804991710,5cyclictest2002947-21cut20:55:191
196479999179,6cyclictest2022255-21awk21:50:200
1964799991715,1cyclictest0-21swapper/020:50:130
1964799991715,1cyclictest0-21swapper/020:24:300
1964799991712,4cyclictest13-21ksoftirqd/000:05:240
1964799991712,3cyclictest1966487-21cron19:15:010
196481199161,7cyclictest141rcu_preempt22:00:213
1964811991614,1cyclictest3056-21PK-Backend21:22:033
1964811991610,4cyclictest2080208-21grep00:35:013
196480899168,7cyclictest2011779-21needreboot21:20:172
196480899168,6cyclictest883-21dbus-daemon21:55:002
1964808991614,1cyclictest0-21swapper/219:50:142
1964808991612,3cyclictest978-21snmpd19:19:322
1964808991611,3cyclictest2061342-21tr23:40:122
196480499165,2cyclictest141rcu_preempt19:30:011
196480499161,4cyclictest26-21ksoftirqd/120:45:121
1964804991614,1cyclictest3056-21PK-Backend23:22:021
1964804991611,3cyclictest20565912chrt23:25:461
196479999161,3cyclictest2058671-21apt-get23:35:000
196481199151,5cyclictest3056-21PK-Backend19:15:023
1964811991510,2cyclictest42-21ksoftirqd/322:15:003
196481199150,6cyclictest3056-21PK-Backend22:05:133
1964808991511,3cyclictest2028884-21tr22:10:132
196480899150,6cyclictest2049073-21grep23:05:192
196480899150,6cyclictest0-21swapper/219:20:172
196479999158,5cyclictest2050432-21basename23:10:150
196481199141,5cyclictest141rcu_preempt20:50:153
196481199140,5cyclictest141rcu_preempt22:20:163
196479999149,3cyclictest2024961-21apt22:00:010
196479999148,5cyclictest2028905-21cat22:10:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional