You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-29 - 05:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Jul 29, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
139099840,50rtkit-daemon0-21swapper/019:07:170
367849328139,13sleep30-21swapper/319:06:523
367843226949,14sleep10-21swapper/119:06:061
367868726843,18sleep20-21swapper/219:09:242
37927342460,5sleep10-21swapper/100:30:221
37870422460,5sleep03056-21PK-Backend00:15:170
37053272330,5sleep03056-21PK-Backend20:25:000
36994182330,1sleep03699421-21wc20:05:190
367889399311,18cyclictest3763300-21apt-get23:10:033
37207952270,12sleep234-21ksoftirqd/221:05:402
37012512270,6chrt0-21swapper/120:10:221
37822032260,4chrt3056-21PK-Backend00:00:203
37653772250,11sleep234-21ksoftirqd/223:15:002
37319952240,3sleep23056-21PK-Backend21:40:012
367889399242,6cyclictest3786640-21apt00:15:143
37794052230,4chrt3779398-21apt-config23:55:001
37659092230,3sleep03056-21PK-Backend23:15:140
37659092230,3sleep03056-21PK-Backend23:15:140
3678893992315,6cyclictest436-20systemd-journal19:15:003
3678890992313,6cyclictest1095-21runrttasks19:30:112
3678883992318,3cyclictest3702831-21ntp_states20:15:211
139099230,16rtkit-daemon3056-21PK-Backend19:47:071
139099230,15rtkit-daemon3056-21PK-Backend19:33:161
37819342220,8sleep013-21ksoftirqd/000:00:170
37782402220,2sleep00-21swapper/023:50:190
3678890992220,1cyclictest0-21swapper/222:21:422
3678890992220,1cyclictest0-21swapper/221:11:542
3678890992212,6cyclictest3761798-21pool-whoopsie23:05:062
367889099220,21cyclictest0-21swapper/220:36:592
367889099220,20cyclictest3056-21PK-Backend22:25:152
3678883992220,1cyclictest0-21swapper/123:40:231
3678883992220,1cyclictest0-21swapper/120:52:051
3678883992220,1cyclictest0-21swapper/120:20:351
3678883992220,1cyclictest0-21swapper/100:24:491
3678880992220,1cyclictest0-21swapper/021:20:530
3678880992220,1cyclictest0-21swapper/020:54:500
3678880992220,1cyclictest0-21swapper/000:32:230
3678880992217,3cyclictest13-21ksoftirqd/000:00:010
367888099220,21cyclictest3056-21PK-Backend21:35:400
367889399210,20cyclictest3056-21PK-Backend21:43:433
367889399210,19cyclictest3056-21PK-Backend22:57:443
367889399210,19cyclictest3056-21PK-Backend20:50:233
367889399210,19cyclictest3056-21PK-Backend00:37:083
367889399210,19cyclictest3056-21PK-Backend00:10:043
367889399210,19cyclictest3056-21PK-Backend00:10:043
367889399210,19cyclictest3056-21PK-Backend00:06:183
367889099218,9cyclictest883-21dbus-daemon23:40:012
3678890992119,1cyclictest3056-21PK-Backend00:39:322
3678890992119,1cyclictest0-21swapper/223:58:092
3678890992119,1cyclictest0-21swapper/223:52:012
3678890992119,1cyclictest0-21swapper/223:45:522
3678890992119,1cyclictest0-21swapper/223:43:282
3678890992119,1cyclictest0-21swapper/223:32:342
3678890992119,1cyclictest0-21swapper/222:11:092
3678890992119,1cyclictest0-21swapper/221:31:482
3678890992119,1cyclictest0-21swapper/221:03:432
3678890992119,1cyclictest0-21swapper/220:14:012
3678890992119,1cyclictest0-21swapper/220:03:332
3678890992119,1cyclictest0-21swapper/219:44:132
3678890992119,1cyclictest0-21swapper/219:38:252
3678890992119,1cyclictest0-21swapper/200:11:132
3678890992119,1cyclictest0-21swapper/200:11:132
3678890992119,1cyclictest0-21swapper/200:04:172
3678890992110,9cyclictest3691676-21/usr/sbin/munin19:45:242
367889099210,20cyclictest0-21swapper/222:33:532
367889099210,20cyclictest0-21swapper/219:57:452
367889099210,20cyclictest0-21swapper/200:15:472
367889099210,20cyclictest0-21swapper/200:05:052
367889099210,19cyclictest3056-21PK-Backend20:50:042
3678883992119,1cyclictest3056-21PK-Backend23:12:311
3678883992119,1cyclictest3056-21PK-Backend23:09:331
3678883992119,1cyclictest3056-21PK-Backend22:04:581
3678883992119,1cyclictest3056-21PK-Backend19:27:081
3678883992119,1cyclictest0-21swapper/122:21:001
3678883992119,1cyclictest0-21swapper/121:43:281
3678883992119,1cyclictest0-21swapper/119:16:471
367888399210,20cyclictest3688495-21expr19:35:121
367888399210,20cyclictest3056-21PK-Backend19:24:381
367888399210,20cyclictest3056-21PK-Backend00:13:351
367888399210,20cyclictest3056-21PK-Backend00:13:351
367888399210,20cyclictest0-21swapper/122:48:521
367888399210,20cyclictest0-21swapper/122:16:071
367888399210,20cyclictest0-21swapper/121:31:121
367888399210,20cyclictest0-21swapper/120:25:481
367888399210,19cyclictest3056-21PK-Backend23:27:171
3678880992119,1cyclictest3056-21PK-Backend20:15:020
3678880992119,1cyclictest0-21swapper/022:55:430
3678880992119,1cyclictest0-21swapper/022:15:410
3678880992119,1cyclictest0-21swapper/021:50:060
3678880992119,1cyclictest0-21swapper/021:10:110
3678880992119,1cyclictest0-21swapper/019:52:040
3678880992119,1cyclictest0-21swapper/019:18:380
3678880992116,3cyclictest13-21ksoftirqd/022:25:000
3678880992110,8cyclictest883-21dbus-daemon23:05:060
367888099210,20cyclictest3751902-21irqstats22:35:140
367888099210,20cyclictest3056-21PK-Backend23:25:380
367888099210,20cyclictest0-21swapper/023:21:520
367888099210,20cyclictest0-21swapper/022:13:310
367888099210,20cyclictest0-21swapper/021:43:510
367888099210,20cyclictest0-21swapper/020:47:270
367888099210,20cyclictest0-21swapper/019:22:500
37450662200,4sleep33056-21PK-Backend22:15:233
367889399203,15cyclictest3700860-21cut20:10:173
3678893992018,1cyclictest3056-21PK-Backend22:00:193
3678893992018,1cyclictest3056-21PK-Backend20:36:573
3678893992012,7cyclictest3705299-21/usr/sbin/munin20:25:003
367889399200,6cyclictest37782392latency23:50:193
367889399200,1cyclictest3056-21PK-Backend22:07:563
367889399200,1cyclictest3056-21PK-Backend21:55:253
367889399200,19cyclictest3056-21PK-Backend22:25:543
367889399200,19cyclictest3056-21PK-Backend20:02:033
367889399200,19cyclictest0-21swapper/323:16:513
367889399200,19cyclictest0-21swapper/323:16:513
367889399200,19cyclictest0-21swapper/320:15:013
367889399200,19cyclictest0-21swapper/319:44:183
367889399200,19cyclictest0-21swapper/300:28:503
3678890992018,1cyclictest0-21swapper/223:15:442
3678890992018,1cyclictest0-21swapper/223:15:432
3678890992018,1cyclictest0-21swapper/222:53:122
3678890992018,1cyclictest0-21swapper/220:30:102
3678890992018,1cyclictest0-21swapper/220:20:032
3678890992018,1cyclictest0-21swapper/219:27:372
3678890992018,1cyclictest0-21swapper/219:22:032
3678890992018,1cyclictest0-21swapper/200:34:122
3678890992017,2cyclictest0-21swapper/200:28:032
3678890992013,5cyclictest3735613-21ls21:50:142
3678890992013,5cyclictest3680972-21cat19:15:022
367889099200,1cyclictest0-21swapper/223:28:352
367889099200,19cyclictest0-21swapper/222:36:512
367889099200,19cyclictest0-21swapper/222:03:182
367889099200,19cyclictest0-21swapper/221:24:112
367889099200,19cyclictest0-21swapper/221:17:212
367889099200,19cyclictest0-21swapper/221:17:212
367889099200,19cyclictest0-21swapper/220:58:152
3678883992018,1cyclictest3056-21PK-Backend21:03:341
3678883992018,1cyclictest0-21swapper/123:04:251
3678883992018,1cyclictest0-21swapper/122:56:221
3678883992018,1cyclictest0-21swapper/120:46:031
3678883992018,1cyclictest0-21swapper/119:59:531
3678883992018,1cyclictest0-21swapper/100:36:541
3678883992018,1cyclictest0-21swapper/100:18:071
3678883992017,2cyclictest3056-21PK-Backend23:34:141
3678883992013,5cyclictest3742224-21perf22:10:001
3678883992012,6cyclictest3730724-21df21:35:131
367888399200,1cyclictest3056-21PK-Backend19:11:191
367888399200,1cyclictest0-21swapper/122:25:011
367888399200,19cyclictest0-21swapper/123:38:471
367888399200,19cyclictest0-21swapper/122:11:131
367888399200,19cyclictest0-21swapper/121:55:111
367888399200,19cyclictest0-21swapper/121:48:561
367888399200,19cyclictest0-21swapper/121:24:221
367888399200,19cyclictest0-21swapper/120:41:571
3678880992018,1cyclictest3056-21PK-Backend22:03:110
3678880992018,1cyclictest0-21swapper/023:42:140
3678880992018,1cyclictest0-21swapper/022:49:010
3678880992018,1cyclictest0-21swapper/022:43:530
3678880992018,1cyclictest0-21swapper/022:08:380
3678880992018,1cyclictest0-21swapper/021:26:540
3678880992018,1cyclictest0-21swapper/020:38:410
3678880992018,1cyclictest0-21swapper/019:25:550
3678880992018,1cyclictest0-21swapper/000:36:080
3678880992018,1cyclictest0-21swapper/000:28:360
367888099201,3cyclictest141rcu_preempt21:00:140
367888099200,1cyclictest3056-21PK-Backend23:14:350
367888099200,1cyclictest0-21swapper/020:31:180
367888099200,1cyclictest0-21swapper/019:14:260
367888099200,1cyclictest0-21swapper/000:21:550
367888099200,19cyclictest3056-21PK-Backend20:56:120
367888099200,19cyclictest0-21swapper/023:33:070
367888099200,19cyclictest0-21swapper/022:26:020
367888099200,19cyclictest0-21swapper/021:32:560
367888099200,19cyclictest0-21swapper/020:40:030
367888099200,19cyclictest0-21swapper/000:10:050
367888099200,19cyclictest0-21swapper/000:10:050
37301582190,4sleep33056-21PK-Backend21:35:013
3678893991917,1cyclictest3056-21PK-Backend21:26:463
3678893991917,1cyclictest0-21swapper/323:55:033
3678893991917,1cyclictest0-21swapper/319:32:223
3678893991917,1cyclictest0-21swapper/319:21:133
3678893991917,1cyclictest0-21swapper/300:30:243
3678893991916,2cyclictest3056-21PK-Backend22:11:273
3678893991914,3cyclictest3771508-21ntp_states23:30:203
3678893991913,5cyclictest42-21ksoftirqd/323:40:203
3678893991913,5cyclictest3762707-21swap23:05:243
3678893991913,5cyclictest3692568-21swap19:45:233
367889399190,1cyclictest3056-21PK-Backend23:00:423
367889399190,1cyclictest3056-21PK-Backend22:45:553
367889399190,1cyclictest3056-21PK-Backend22:41:343
367889399190,1cyclictest3056-21PK-Backend22:37:163
367889399190,1cyclictest3056-21PK-Backend20:06:293
367889399190,1cyclictest3056-21PK-Backend19:57:373
367889399190,1cyclictest0-21swapper/323:48:553
367889399190,1cyclictest0-21swapper/323:37:253
367889399190,1cyclictest0-21swapper/321:09:153
367889399190,1cyclictest0-21swapper/320:25:273
367889399190,18cyclictest3056-21PK-Backend21:18:013
367889399190,18cyclictest3056-21PK-Backend21:18:013
367889399190,18cyclictest3056-21PK-Backend19:52:373
367889399190,18cyclictest0-21swapper/321:35:313
367889399190,18cyclictest0-21swapper/321:01:523
367889399190,18cyclictest0-21swapper/320:57:463
367889399190,18cyclictest0-21swapper/319:18:433
367889099198,8cyclictest3707031-21apt-get20:30:002
3678890991917,1cyclictest0-21swapper/222:45:432
3678890991917,1cyclictest0-21swapper/222:42:452
3678890991917,1cyclictest0-21swapper/222:18:122
3678890991916,2cyclictest3056-21PK-Backend22:07:372
3678890991912,5cyclictest3759279-21cat22:55:232
367889099190,1cyclictest0-21swapper/223:04:292
367889099190,1cyclictest0-21swapper/221:40:472
367889099190,1cyclictest0-21swapper/220:07:252
3678883991917,1cyclictest3056-21PK-Backend21:06:171
3678883991917,1cyclictest0-21swapper/123:58:341
3678883991917,1cyclictest0-21swapper/122:50:271
3678883991917,1cyclictest0-21swapper/120:35:561
3678883991917,1cyclictest0-21swapper/120:08:111
3678883991913,5cyclictest3708823-21sshd20:32:401
3678883991913,4cyclictest3723602-21if_enp2s021:15:191
3678883991913,4cyclictest3723602-21if_enp2s021:15:191
3678883991912,5cyclictest3697155-21expr20:00:111
367888399190,1cyclictest3056-21PK-Backend00:28:351
367888399190,1cyclictest0-21swapper/121:10:561
367888399190,1cyclictest0-21swapper/119:50:261
367888399190,1cyclictest0-21swapper/100:02:191
367888399190,18cyclictest3056-21PK-Backend23:21:571
367888399190,18cyclictest0-21swapper/122:39:251
367888399190,18cyclictest0-21swapper/100:06:521
3678880991917,1cyclictest0-21swapper/022:33:050
3678880991917,1cyclictest0-21swapper/021:08:490
3678880991917,1cyclictest0-21swapper/019:42:040
3678880991913,4cyclictest3756787-21grep22:50:120
3678880991911,6cyclictest3688956-21ntpq19:35:180
367888099190,1cyclictest0-21swapper/023:00:500
367888099190,1cyclictest0-21swapper/020:28:000
139099190,13rtkit-daemon3056-21PK-Backend23:47:581
3678893991816,1cyclictest3056-21PK-Backend21:13:213
3678893991816,1cyclictest3056-21PK-Backend19:39:523
3678893991816,1cyclictest3056-21PK-Backend00:22:073
3678893991816,1cyclictest0-21swapper/323:28:543
3678893991816,1cyclictest0-21swapper/320:30:563
367889399181,15cyclictest3685498-21/usr/sbin/munin19:25:193
367889399180,5cyclictest3767693-21hddtemp_smartct23:20:173
367889099184,4cyclictest34-21ksoftirqd/221:55:162
3678890991816,1cyclictest0-21swapper/223:21:392
3678890991816,1cyclictest0-21swapper/220:45:582
3678890991816,1cyclictest0-21swapper/219:51:232
3678890991811,5cyclictest3734315-21sed21:45:182
3678883991816,1cyclictest0-21swapper/123:15:281
3678883991816,1cyclictest0-21swapper/123:15:281
3678883991816,1cyclictest0-21swapper/122:30:471
367888399180,1cyclictest3056-21PK-Backend19:42:081
367888099183,2cyclictest141rcu_preempt21:50:000
3678880991816,1cyclictest3056-21PK-Backend23:46:470
3678880991816,1cyclictest0-21swapper/020:03:260
3678880991816,1cyclictest0-21swapper/019:56:290
3678880991813,4cyclictest13-21ksoftirqd/021:20:000
3678880991813,4cyclictest13-21ksoftirqd/021:20:000
3678893991715,1cyclictest3056-21PK-Backend21:20:443
3678893991714,2cyclictest3056-21PK-Backend22:54:473
3678893991710,5cyclictest3712936-21grep20:45:143
3678890991715,1cyclictest0-21swapper/220:18:412
3678890991715,1cyclictest0-21swapper/219:11:352
3678890991715,1cyclictest0-21swapper/200:22:432
3678883991715,1cyclictest3056-21PK-Backend20:57:321
3678883991715,1cyclictest0-21swapper/122:42:231
3678883991715,1cyclictest0-21swapper/121:25:101
3678880991715,1cyclictest0-21swapper/021:56:210
3678880991715,1cyclictest0-21swapper/020:11:100
3678880991715,1cyclictest0-21swapper/019:47:040
3678893991614,1cyclictest0-21swapper/322:20:263
3678893991613,2cyclictest3056-21PK-Backend21:51:203
3678893991610,4cyclictest3712487-21cron20:45:003
3678890991614,1cyclictest0-21swapper/221:26:202
367888399161,11cyclictest3735556-21apt21:50:141
3678880991613,2cyclictest3056-21PK-Backend23:35:310
3678880991611,3cyclictest2156-21gnome-shell19:32:090
3678890991513,1cyclictest0-21swapper/220:40:312
367888099158,5cyclictest3783788-21wc00:05:210
3678893991410,3cyclictest3056-21PK-Backend22:30:503
3678893991310,2cyclictest3056-21PK-Backend21:49:583
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional