You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-13 - 07:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Sep 13, 2026 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29897821000,5sleep33056-21PK-Backend23:45:223
20001327639,14sleep10-21swapper/119:08:361
20009426839,24sleep00-21swapper/019:09:390
19906826845,15sleep30-21swapper/319:05:163
20007526645,14sleep20-21swapper/219:09:262
2022652550,4sleep20-21swapper/219:15:012
2823452470,2sleep20-21swapper/223:00:142
2965002430,3sleep10-21swapper/123:40:151
2212532430,2sleep20-21swapper/220:05:232
2758332320,2chrt0-21swapper/022:40:220
139099280,4rtkit-daemon1389-21rtkit-daemon20:24:153
139099280,4rtkit-daemon1389-21rtkit-daemon20:16:153
2176772270,4chrt0-21swapper/219:55:242
139099270,4rtkit-daemon1389-21rtkit-daemon22:39:253
139099270,4rtkit-daemon1389-21rtkit-daemon22:39:253
139099260,4rtkit-daemon1389-21rtkit-daemon23:35:513
139099260,4rtkit-daemon1389-21rtkit-daemon00:22:013
200281992514,8cyclictest883-21dbus-daemon19:39:593
139099250,4rtkit-daemon1389-21rtkit-daemon23:32:533
139099250,4rtkit-daemon1389-21rtkit-daemon21:58:383
139099250,4rtkit-daemon1389-21rtkit-daemon00:19:563
2772142240,7sleep013-21ksoftirqd/022:45:200
20028199242,19cyclictest212476-21python319:40:213
20028199241,21cyclictest227778-21/usr/sbin/munin20:25:193
139099240,4rtkit-daemon1389-21rtkit-daemon23:26:162
139099240,4rtkit-daemon1389-21rtkit-daemon00:25:453
20027699230,21cyclictest241909-21memory21:05:172
200274992320,2cyclictest3056-21PK-Backend21:54:531
139099230,4rtkit-daemon1389-21rtkit-daemon23:24:232
3002462220,3chrt0-21swapper/223:50:182
20028199227,13cyclictest265636-21latency_hist22:15:003
20028199222,16cyclictest1095-21runrttasks20:51:433
200281992220,1cyclictest0-21swapper/322:24:503
20028199221,9cyclictest261049-21chrt22:00:163
200281992213,7cyclictest253448-21cron21:40:003
200281992213,7cyclictest253448-21cron21:39:593
200274992220,1cyclictest3056-21PK-Backend20:38:291
200274992220,1cyclictest3056-21PK-Backend20:38:291
200274992219,2cyclictest3056-21PK-Backend20:33:501
200274992219,2cyclictest3056-21PK-Backend20:33:491
200274992214,6cyclictest883-21dbus-daemon22:47:101
20027499220,21cyclictest0-21swapper/123:24:571
20027499220,20cyclictest3056-21PK-Backend21:02:021
20027499220,20cyclictest3056-21PK-Backend19:22:101
200270992214,5cyclictest214794-21cron19:50:000
20027099221,16cyclictest1788620-21WebKitWebProces23:45:040
200281992119,1cyclictest3056-21PK-Backend19:52:073
200281992119,1cyclictest3056-21PK-Backend19:49:433
200281992119,1cyclictest3056-21PK-Backend19:18:283
200281992119,1cyclictest0-21swapper/322:29:033
200281992119,1cyclictest0-21swapper/321:32:513
200281992119,1cyclictest0-21swapper/321:32:513
200281992119,1cyclictest0-21swapper/320:56:083
200281992119,1cyclictest0-21swapper/319:31:313
20028199211,17cyclictest306527-21munin-run00:10:003
20028199211,17cyclictest222171-21cpuspeed_turbos20:10:123
20028199210,20cyclictest0-21swapper/321:53:263
20028199210,20cyclictest0-21swapper/321:44:553
20028199210,19cyclictest3056-21PK-Backend23:55:483
20028199210,19cyclictest3056-21PK-Backend19:10:153
200276992114,5cyclictest263552-21sshd22:06:252
200276992114,5cyclictest239558-21date21:00:012
20027699210,20cyclictest0-21swapper/222:27:172
20027699210,20cyclictest0-21swapper/220:47:412
20027699210,20cyclictest0-21swapper/220:10:152
200274992119,1cyclictest3056-21PK-Backend22:08:181
200274992119,1cyclictest3056-21PK-Backend19:30:011
200274992119,1cyclictest0-21swapper/123:26:461
200274992119,1cyclictest0-21swapper/121:55:411
200274992118,2cyclictest3056-21PK-Backend22:15:501
200274992118,2cyclictest3056-21PK-Backend21:46:021
200274992118,2cyclictest3056-21PK-Backend20:52:281
200274992118,2cyclictest3056-21PK-Backend20:27:331
200274992113,6cyclictest215943-21python319:50:221
20027499210,20cyclictest0-21swapper/119:19:401
20027499210,20cyclictest0-21swapper/100:13:451
20027499210,20cyclictest0-21swapper/100:06:151
20027499210,19cyclictest3056-21PK-Backend20:41:461
20027499210,19cyclictest3056-21PK-Backend20:15:381
20027499210,19cyclictest3056-21PK-Backend00:26:281
200270992119,1cyclictest0-21swapper/022:11:410
200270992119,1cyclictest0-21swapper/021:41:120
200270992118,2cyclictest0-21swapper/021:35:190
200270992118,2cyclictest0-21swapper/021:35:180
200270992115,4cyclictest2156-21gnome-shell00:16:100
20027099210,20cyclictest0-21swapper/023:32:460
20027099210,19cyclictest3056-21PK-Backend20:35:230
20027099210,19cyclictest3056-21PK-Backend20:35:220
139099210,3rtkit-daemon1389-21rtkit-daemon22:59:173
2456372200,4sleep00-21swapper/021:15:220
2157152200,5sleep20-21swapper/219:50:182
20028199209,7cyclictest314104-21kworker/u8:000:30:263
20028199209,7cyclictest314104-21kworker/u8:000:30:263
200281992018,1cyclictest3056-21PK-Backend23:54:003
200281992018,1cyclictest3056-21PK-Backend23:22:023
200281992018,1cyclictest3056-21PK-Backend23:08:093
200281992018,1cyclictest3056-21PK-Backend23:01:063
200281992018,1cyclictest3056-21PK-Backend19:55:183
200281992018,1cyclictest3056-21PK-Backend00:36:173
200281992018,1cyclictest3056-21PK-Backend00:36:173
200281992018,1cyclictest0-21swapper/322:32:273
200281992018,1cyclictest0-21swapper/320:36:493
200281992018,1cyclictest0-21swapper/320:36:483
200281992018,1cyclictest0-21swapper/319:28:353
200281992017,2cyclictest3056-21PK-Backend23:16:013
200281992013,5cyclictest230408-21latency_hist20:35:003
200281992013,5cyclictest230408-21latency_hist20:35:003
200281992013,5cyclictest205136-21ntpq19:20:223
200281992012,6cyclictest436-20systemd-journal23:44:553
20028199200,1cyclictest3056-21PK-Backend23:28:033
20028199200,1cyclictest3056-21PK-Backend21:14:343
20028199200,19cyclictest3056-21PK-Backend23:14:113
20028199200,19cyclictest3056-21PK-Backend00:02:033
200276992018,1cyclictest0-21swapper/223:38:012
200276992018,1cyclictest0-21swapper/223:07:052
200276992018,1cyclictest0-21swapper/222:58:002
200276992018,1cyclictest0-21swapper/220:51:122
200276992018,1cyclictest0-21swapper/219:11:142
200276992013,6cyclictest305634-21memory00:05:182
200276992012,6cyclictest276245-21ls22:45:002
20027699200,19cyclictest0-21swapper/222:23:522
20027699200,19cyclictest0-21swapper/221:57:022
20027699200,19cyclictest0-21swapper/221:24:592
20027699200,19cyclictest0-21swapper/220:23:472
20027699200,19cyclictest0-21swapper/219:42:032
20027699200,19cyclictest0-21swapper/200:32:092
20027699200,19cyclictest0-21swapper/200:32:082
20027699200,19cyclictest0-21swapper/200:20:262
20027499207,11cyclictest274541-21unin-run22:40:011
20027499207,11cyclictest274541-21unin-run22:40:001
200274992018,1cyclictest3056-21PK-Backend23:38:441
200274992018,1cyclictest3056-21PK-Backend22:12:041
200274992018,1cyclictest3056-21PK-Backend19:36:311
200274992018,1cyclictest0-21swapper/123:14:291
200274992018,1cyclictest0-21swapper/121:09:391
200274992018,1cyclictest0-21swapper/119:57:051
200274992018,1cyclictest0-21swapper/100:37:581
200274992018,1cyclictest0-21swapper/100:37:571
200274992017,2cyclictest3056-21PK-Backend00:21:141
200274992013,5cyclictest298073-21seq23:45:101
200274992013,5cyclictest278976-21ntp_kernel_pll_22:50:241
200274992010,8cyclictest276274-21unin-run22:45:011
20027499200,1cyclictest0-21swapper/123:19:431
20027499200,1cyclictest0-21swapper/119:45:231
20027499200,19cyclictest3056-21PK-Backend21:44:261
20027499200,19cyclictest3056-21PK-Backend20:20:301
20027499200,19cyclictest0-21swapper/123:07:271
20027499200,19cyclictest0-21swapper/121:26:551
20027499200,19cyclictest0-21swapper/121:15:191
20027499200,19cyclictest0-21swapper/121:14:311
20027499200,18cyclictest3056-21PK-Backend00:16:351
200270992018,1cyclictest0-21swapper/022:23:380
200270992018,1cyclictest0-21swapper/022:18:370
200270992018,1cyclictest0-21swapper/020:07:440
200270992018,1cyclictest0-21swapper/019:36:480
200270992018,1cyclictest0-21swapper/019:33:030
200270992018,1cyclictest0-21swapper/019:27:560
200270992018,1cyclictest0-21swapper/019:23:380
200270992012,6cyclictest436-20systemd-journal23:54:240
20027099200,1cyclictest3056-21PK-Backend23:40:170
20027099200,19cyclictest0-21swapper/022:57:030
20027099200,19cyclictest0-21swapper/021:49:310
20027099200,19cyclictest0-21swapper/020:58:340
20027099200,19cyclictest0-21swapper/019:16:070
2777442190,3sleep20-21swapper/222:47:102
20028199192,13cyclictest241332-21apt21:05:123
200281991918,1cyclictest0-21swapper/321:04:333
200281991917,1cyclictest3056-21PK-Backend00:12:243
200281991916,2cyclictest3056-21PK-Backend22:52:153
200281991916,2cyclictest3056-21PK-Backend20:07:003
20028199191,15cyclictest232827-21cpuspeed_turbos20:40:113
20028199190,1cyclictest0-21swapper/320:46:223
20028199190,18cyclictest3056-21PK-Backend21:18:533
200276991917,1cyclictest0-21swapper/223:47:202
200276991917,1cyclictest0-21swapper/223:40:512
200276991917,1cyclictest0-21swapper/223:33:222
200276991917,1cyclictest0-21swapper/222:19:402
200276991917,1cyclictest0-21swapper/221:42:232
200276991917,1cyclictest0-21swapper/220:29:432
200276991917,1cyclictest0-21swapper/219:30:492
200276991917,1cyclictest0-21swapper/219:23:312
200276991914,3cyclictest313455-21idleruntime-cro00:29:592
200276991912,6cyclictest258788-21sed21:55:002
200276991912,5cyclictest255872-21ls21:45:162
20027699190,1cyclictest0-21swapper/220:37:192
20027699190,1cyclictest0-21swapper/220:37:192
20027699190,1cyclictest0-21swapper/220:17:182
20027699190,1cyclictest0-21swapper/220:03:132
20027699190,1cyclictest0-21swapper/200:02:082
20027699190,18cyclictest0-21swapper/221:33:162
20027699190,18cyclictest0-21swapper/221:33:162
20027699190,18cyclictest0-21swapper/220:34:362
20027699190,18cyclictest0-21swapper/220:34:362
200274991917,1cyclictest3056-21PK-Backend21:37:091
200274991917,1cyclictest3056-21PK-Backend21:37:091
200274991917,1cyclictest0-21swapper/123:50:391
200274991917,1cyclictest0-21swapper/123:02:131
200274991917,1cyclictest0-21swapper/121:32:031
200274991917,1cyclictest0-21swapper/121:32:031
200274991917,1cyclictest0-21swapper/121:23:571
200274991916,2cyclictest3056-21PK-Backend22:24:341
200274991916,2cyclictest3056-21PK-Backend22:02:241
200274991912,5cyclictest220400-21expr20:05:131
200274991911,6cyclictest303376-21awk00:00:131
20027499190,1cyclictest3056-21PK-Backend19:41:381
20027499190,1cyclictest0-21swapper/123:58:441
20027499190,1cyclictest0-21swapper/123:31:261
20027499190,1cyclictest0-21swapper/122:56:451
20027499190,18cyclictest3056-21PK-Backend20:46:261
200270991917,1cyclictest3056-21PK-Backend22:05:330
200270991917,1cyclictest0-21swapper/023:58:410
200270991917,1cyclictest0-21swapper/023:15:500
200270991917,1cyclictest0-21swapper/023:11:240
200270991917,1cyclictest0-21swapper/022:33:380
200270991917,1cyclictest0-21swapper/021:14:500
200270991917,1cyclictest0-21swapper/020:46:510
200270991917,1cyclictest0-21swapper/020:34:000
200270991917,1cyclictest0-21swapper/020:34:000
200270991917,1cyclictest0-21swapper/019:56:020
200270991917,1cyclictest0-21swapper/019:41:210
200270991917,1cyclictest0-21swapper/000:32:550
200270991917,1cyclictest0-21swapper/000:32:550
200270991917,1cyclictest0-21swapper/000:23:030
200270991917,1cyclictest0-21swapper/000:13:420
200270991910,7cyclictest883-21dbus-daemon00:40:010
200270991910,7cyclictest883-21dbus-daemon00:40:000
20027099190,1cyclictest3056-21PK-Backend21:55:400
20027099190,1cyclictest0-21swapper/023:35:360
20027099190,1cyclictest0-21swapper/023:05:080
20027099190,1cyclictest0-21swapper/021:21:180
20027099190,1cyclictest0-21swapper/020:53:400
20027099190,1cyclictest0-21swapper/020:16:100
20027099190,1cyclictest0-21swapper/000:04:090
20027099190,18cyclictest0-21swapper/023:26:170
20027099190,18cyclictest0-21swapper/023:21:030
20027099190,18cyclictest0-21swapper/021:31:330
20027099190,18cyclictest0-21swapper/021:31:320
20027099190,18cyclictest0-21swapper/020:13:590
20027099190,18cyclictest0-21swapper/019:52:030
20027099190,18cyclictest0-21swapper/000:29:170
200281991816,1cyclictest3056-21PK-Backend21:26:093
200281991815,2cyclictest3056-21PK-Backend20:01:333
20028199181,15cyclictest265853-21apt22:15:123
20028199180,16cyclictest256201-21ntp_states21:45:203
200276991816,1cyclictest3056-21PK-Backend23:56:412
200276991816,1cyclictest0-21swapper/223:10:432
200276991816,1cyclictest0-21swapper/222:52:322
200276991816,1cyclictest0-21swapper/221:26:342
200276991816,1cyclictest0-21swapper/200:17:352
200276991813,3cyclictest886-21NetworkManager19:40:012
200276991812,5cyclictest274013-21sed22:35:242
200276991812,5cyclictest274013-21sed22:35:232
200276991812,4cyclictest265030-21cat22:10:222
200276991810,6cyclictest436-20systemd-journal22:35:002
20027699180,16cyclictest206156-21dpkg19:25:142
200274991816,1cyclictest3056-21PK-Backend00:30:541
200274991816,1cyclictest3056-21PK-Backend00:30:541
200274991811,5cyclictest200852-21forks19:10:171
20027099189,6cyclictest436-20systemd-journal23:05:000
200270991816,1cyclictest0-21swapper/022:27:510
200270991816,1cyclictest0-21swapper/020:26:570
200270991812,4cyclictest201972-21latency_hist19:15:010
200281991715,1cyclictest3056-21PK-Backend22:44:503
200281991711,4cyclictest263146-21awk22:05:203
200276991715,1cyclictest0-21swapper/223:16:582
200276991715,1cyclictest0-21swapper/221:01:462
200276991715,1cyclictest0-21swapper/200:39:252
200276991715,1cyclictest0-21swapper/200:39:252
200276991711,5cyclictest214798-21cron19:49:592
200276991710,5cyclictest252298-21expr21:35:152
200276991710,5cyclictest252298-21expr21:35:142
200274991715,1cyclictest3056-21PK-Backend20:56:341
200274991714,2cyclictest3056-21PK-Backend19:27:511
200274991712,3cyclictest272779-21turbostat.cron22:34:591
200274991710,5cyclictest222071-21cat20:10:131
200270991712,3cyclictest1095-21runrttasks20:00:090
200270991711,4cyclictest273210-21tr22:35:140
200270991711,4cyclictest273210-21tr22:35:140
200270991711,3cyclictest242793-21sh21:10:010
20027699169,5cyclictest245312-21wc21:15:182
20027699168,6cyclictest261728-21users22:00:222
200276991614,1cyclictest0-21swapper/221:10:242
200276991612,3cyclictest306977-21cpuspeed_turbos00:10:172
200274991614,1cyclictest0-21swapper/120:01:101
200274991611,3cyclictest269460-21cpuspeed_turbos22:25:111
20027099169,5cyclictest226360-21grep20:20:240
20027099167,7cyclictest248911-21needreboot21:25:160
200270991614,1cyclictest0-21swapper/021:00:430
200270991610,4cyclictest305852-21ps00:05:200
20028199159,4cyclictest42-21ksoftirqd/322:50:003
20027099156,7cyclictest260762-21cat22:00:130
20027099151,10cyclictest278266-21apt22:50:180
200281991412,1cyclictest3056-21PK-Backend21:21:503
20027699145,8cyclictest2174-21JS10:27:482
20027099147,5cyclictest233227-21cat20:40:150
200270991412,1cyclictest0-21swapper/021:51:550
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional