You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-09 - 23:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sat May 09, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
418703827740,14sleep00-21swapper/007:06:030
418676427549,18sleep30-21swapper/307:05:263
418701526343,14sleep20-21swapper/207:05:462
418626826340,15sleep10-21swapper/107:05:171
64562530,3sleep33056-21PK-Backend07:45:003
336882430,4sleep03056-21PK-Backend09:00:190
4187490992816,10cyclictest82637-21python311:20:210
757572270,3sleep30-21swapper/311:00:213
139099270,4rtkit-daemon1389-21rtkit-daemon09:42:503
139099260,4rtkit-daemon1389-21rtkit-daemon10:17:123
139099260,4rtkit-daemon1389-21rtkit-daemon08:36:272
139099250,4rtkit-daemon1389-21rtkit-daemon08:38:553
139099250,4rtkit-daemon1389-21rtkit-daemon07:21:212
1095512250,3sleep2109533-21apt-get12:39:592
418750499231,6cyclictest436-20systemd-journal10:25:243
4187504992316,6cyclictest30202-21/usr/sbin/munin08:50:183
4187500992319,2cyclictest76565-21apt11:05:122
418750099230,21cyclictest8906-21/usr/sbin/munin07:50:162
418750099230,21cyclictest28082-21expr08:45:132
418749599233,5cyclictest26-21ksoftirqd/112:25:041
418749599230,21cyclictest7087-21hddtemp_smartct07:45:151
418749099232,19cyclictest436-20systemd-journal10:39:590
4187504992220,1cyclictest3056-21PK-Backend12:31:003
4187504992220,1cyclictest3056-21PK-Backend10:51:383
4187504992220,1cyclictest0-21swapper/309:08:223
4187504992220,1cyclictest0-21swapper/308:46:333
4187504992219,2cyclictest3056-21PK-Backend11:17:463
4187504992219,2cyclictest3056-21PK-Backend09:31:133
4187504992219,2cyclictest3056-21PK-Backend08:22:473
418750499220,20cyclictest3056-21PK-Backend11:41:123
418750499220,20cyclictest3056-21PK-Backend10:34:353
418750499220,20cyclictest3056-21PK-Backend10:34:353
4187500992220,1cyclictest0-21swapper/211:33:312
4187500992220,1cyclictest0-21swapper/211:24:032
4187500992220,1cyclictest0-21swapper/210:55:032
4187500992220,1cyclictest0-21swapper/209:33:382
418750099220,21cyclictest100569-21perf12:15:002
4187495992215,5cyclictest883-21dbus-daemon07:58:021
4187495992214,6cyclictest87553-21memory11:35:171
4187490992216,5cyclictest62160-21idleruntime-cro10:25:000
4187490992213,6cyclictest5332-21sshd07:40:150
139099220,4rtkit-daemon1389-21rtkit-daemon11:35:462
418750499219,10cyclictest42-21ksoftirqd/312:25:023
4187504992119,1cyclictest3056-21PK-Backend12:17:233
4187504992119,1cyclictest3056-21PK-Backend10:42:113
4187504992119,1cyclictest3056-21PK-Backend09:24:113
4187504992119,1cyclictest3056-21PK-Backend08:08:473
4187504992119,1cyclictest0-21swapper/312:13:093
4187504992119,1cyclictest0-21swapper/309:58:583
4187504992119,1cyclictest0-21swapper/309:01:193
4187504992119,1cyclictest0-21swapper/307:52:323
4187504992113,6cyclictest26754-21ntp_states08:40:183
418750499210,20cyclictest0-21swapper/308:32:263
418750499210,20cyclictest0-21swapper/308:11:043
4187500992119,1cyclictest0-21swapper/212:06:362
4187500992119,1cyclictest0-21swapper/210:19:152
4187500992119,1cyclictest0-21swapper/209:21:092
4187500992119,1cyclictest0-21swapper/208:51:142
4187500992119,1cyclictest0-21swapper/208:43:562
4187500992119,1cyclictest0-21swapper/208:04:442
4187500992119,1cyclictest0-21swapper/207:28:482
418750099210,20cyclictest3056-21PK-Backend08:25:322
418750099210,20cyclictest0-21swapper/211:04:102
418750099210,19cyclictest0-21swapper/208:20:112
4187495992119,1cyclictest0-21swapper/111:00:021
4187495992119,1cyclictest0-21swapper/110:35:421
4187495992114,5cyclictest1-21systemd07:40:001
418749599210,20cyclictest0-21swapper/107:44:141
418749099210,20cyclictest0-21swapper/010:25:420
418749099210,20cyclictest0-21swapper/009:39:050
418749099210,20cyclictest0-21swapper/009:39:050
418749099210,20cyclictest0-21swapper/009:28:090
418749099210,20cyclictest0-21swapper/009:22:550
418749099210,20cyclictest0-21swapper/008:25:500
724142200,6sleep20-21swapper/210:50:202
4187504992018,1cyclictest3056-21PK-Backend11:36:593
4187504992018,1cyclictest3056-21PK-Backend10:49:003
4187504992018,1cyclictest0-21swapper/310:10:493
4187504992018,1cyclictest0-21swapper/310:08:463
4187504992018,1cyclictest0-21swapper/308:58:423
4187504992018,1cyclictest0-21swapper/307:10:063
418750499200,1cyclictest3056-21PK-Backend11:34:233
418750499200,1cyclictest3056-21PK-Backend10:21:083
418750499200,19cyclictest3056-21PK-Backend12:00:453
418750499200,19cyclictest3056-21PK-Backend11:47:483
418750499200,19cyclictest3056-21PK-Backend07:33:313
418750499200,19cyclictest0-21swapper/307:49:003
418750499200,18cyclictest3056-21PK-Backend12:35:133
418750499200,18cyclictest3056-21PK-Backend07:57:053
4187500992018,1cyclictest3056-21PK-Backend09:40:222
4187500992018,1cyclictest0-21swapper/212:23:382
4187500992018,1cyclictest0-21swapper/212:16:492
4187500992018,1cyclictest0-21swapper/212:03:112
4187500992018,1cyclictest0-21swapper/211:48:102
4187500992018,1cyclictest0-21swapper/211:26:412
4187500992018,1cyclictest0-21swapper/211:19:512
4187500992018,1cyclictest0-21swapper/210:26:042
4187500992018,1cyclictest0-21swapper/210:02:182
4187500992018,1cyclictest0-21swapper/209:57:242
4187500992018,1cyclictest0-21swapper/208:59:392
4187500992018,1cyclictest0-21swapper/208:07:002
4187500992017,2cyclictest0-21swapper/209:18:322
4187500992013,5cyclictest4189427-21forks07:15:012
418750099200,1cyclictest0-21swapper/210:43:222
418750099200,1cyclictest0-21swapper/209:52:322
418750099200,1cyclictest0-21swapper/207:56:052
418750099200,19cyclictest0-21swapper/211:41:202
418750099200,19cyclictest0-21swapper/210:38:082
418750099200,19cyclictest0-21swapper/210:32:542
418750099200,19cyclictest0-21swapper/210:32:532
418750099200,19cyclictest0-21swapper/209:04:532
418750099200,19cyclictest0-21swapper/208:31:062
418750099200,19cyclictest0-21swapper/208:12:332
418749599209,9cyclictest53531-21unin-run10:00:011
4187495992018,1cyclictest0-21swapper/107:23:581
4187495992013,5cyclictest95444-21date12:00:001
4187495992013,5cyclictest436-20systemd-journal11:47:231
4187495992013,5cyclictest43275-21wc09:30:011
4187495992012,6cyclictest466-21sshd07:25:231
4187495992012,6cyclictest46006-21cat09:35:221
4187495992012,6cyclictest46006-21cat09:35:211
418749599200,19cyclictest3056-21PK-Backend12:11:141
418749599200,19cyclictest3056-21PK-Backend09:01:341
418749599200,19cyclictest3056-21PK-Backend08:40:521
418749599200,19cyclictest0-21swapper/112:32:021
418749599200,19cyclictest0-21swapper/111:51:121
418749599200,19cyclictest0-21swapper/110:45:161
4187490992018,1cyclictest0-21swapper/010:18:180
4187490992018,1cyclictest0-21swapper/008:34:420
4187490992018,1cyclictest0-21swapper/007:37:500
4187490992018,1cyclictest0-21swapper/007:13:040
418749099200,1cyclictest3056-21PK-Backend12:21:550
418749099200,1cyclictest0-21swapper/008:48:490
418749099200,1cyclictest0-21swapper/007:50:010
418749099200,19cyclictest3056-21PK-Backend09:08:090
418749099200,19cyclictest0-21swapper/012:37:080
418749099200,19cyclictest0-21swapper/011:54:310
418749099200,19cyclictest0-21swapper/011:35:300
418749099200,19cyclictest0-21swapper/010:51:100
418749099200,19cyclictest0-21swapper/010:45:090
418749099200,19cyclictest0-21swapper/010:42:310
418749099200,19cyclictest0-21swapper/010:30:420
418749099200,19cyclictest0-21swapper/010:30:420
418749099200,19cyclictest0-21swapper/009:53:510
418749099200,19cyclictest0-21swapper/007:47:450
328272200,2sleep13056-21PK-Backend09:00:011
139099200,16rtkit-daemon1788643-21eadedCompositor07:44:522
420432190,9chrt34-21ksoftirqd/209:25:162
4187504991917,1cyclictest3056-21PK-Backend12:08:093
4187504991917,1cyclictest3056-21PK-Backend09:36:423
4187504991917,1cyclictest3056-21PK-Backend09:36:423
4187504991917,1cyclictest3056-21PK-Backend08:29:093
4187504991917,1cyclictest3056-21PK-Backend07:37:033
4187504991917,1cyclictest0-21swapper/312:28:243
4187504991916,2cyclictest3056-21PK-Backend11:29:233
4187504991916,2cyclictest3056-21PK-Backend10:37:583
418750499190,1cyclictest3056-21PK-Backend09:26:473
418750499190,1cyclictest0-21swapper/309:52:153
4187500991917,1cyclictest0-21swapper/211:58:592
4187500991917,1cyclictest0-21swapper/211:54:452
4187500991917,1cyclictest0-21swapper/210:22:392
4187500991917,1cyclictest0-21swapper/208:15:032
418750099190,1cyclictest0-21swapper/212:34:552
418750099190,1cyclictest0-21swapper/210:45:102
418750099190,1cyclictest0-21swapper/210:14:222
418750099190,1cyclictest0-21swapper/209:45:012
4187495991917,1cyclictest0-21swapper/112:36:021
4187495991917,1cyclictest0-21swapper/112:18:571
4187495991917,1cyclictest0-21swapper/110:28:331
4187495991917,1cyclictest0-21swapper/110:17:521
4187495991917,1cyclictest0-21swapper/109:18:431
4187495991915,2cyclictest37871-21sh09:15:011
4187495991913,5cyclictest26-21ksoftirqd/110:50:111
4187495991913,4cyclictest88829-21grep11:40:121
4187495991912,5cyclictest56216-21ntpq10:05:181
4187495991912,5cyclictest4188241-21cat07:10:171
4187495991912,5cyclictest23269-21ntpq08:30:191
4187495991912,5cyclictest21023-21cat08:25:131
4187495991912,5cyclictest102569-21cat12:20:081
4187495991911,6cyclictest883-21dbus-daemon07:55:001
4187495991911,6cyclictest1139-21sshd07:30:081
418749599190,1cyclictest3056-21PK-Backend11:31:461
418749599190,1cyclictest3056-21PK-Backend11:09:351
418749599190,1cyclictest0-21swapper/112:04:511
418749599190,1cyclictest0-21swapper/110:23:121
418749599190,18cyclictest3056-21PK-Backend09:09:521
418749599190,18cyclictest0-21swapper/111:25:371
418749599190,18cyclictest0-21swapper/111:13:211
4187490991917,1cyclictest3056-21PK-Backend11:00:020
4187490991917,1cyclictest3056-21PK-Backend07:16:420
4187490991917,1cyclictest0-21swapper/012:08:290
4187490991917,1cyclictest0-21swapper/011:42:200
4187490991917,1cyclictest0-21swapper/011:27:540
4187490991917,1cyclictest0-21swapper/009:40:060
4187490991917,1cyclictest0-21swapper/008:17:260
4187490991917,1cyclictest0-21swapper/008:06:300
4187490991917,1cyclictest0-21swapper/008:00:080
4187490991917,1cyclictest0-21swapper/007:31:010
4187490991916,2cyclictest3056-21PK-Backend10:06:290
4187490991916,2cyclictest3056-21PK-Backend10:01:360
4187490991914,4cyclictest91933-21systemd-run11:50:010
4187490991913,5cyclictest101386-21cat12:15:160
4187490991913,4cyclictest2156-21gnome-shell11:12:080
4187490991913,4cyclictest2156-21gnome-shell08:55:080
4187490991911,7cyclictest53487-21cron10:00:010
418749099190,1cyclictest3056-21PK-Backend11:07:050
418749099190,1cyclictest0-21swapper/012:14:170
418749099190,1cyclictest0-21swapper/009:19:310
4187504991816,1cyclictest3056-21PK-Backend11:59:593
4187504991816,1cyclictest3056-21PK-Backend10:56:043
4187504991816,1cyclictest3056-21PK-Backend09:46:483
4187504991816,1cyclictest3056-21PK-Backend07:22:513
4187504991816,1cyclictest0-21swapper/310:03:523
4187504991816,1cyclictest0-21swapper/307:18:113
4187504991815,2cyclictest3056-21PK-Backend08:15:373
4187504991810,6cyclictest79755-21cron11:15:003
418750499180,16cyclictest37158-21ntp_states09:10:203
4187500991816,1cyclictest0-21swapper/210:05:232
4187500991816,1cyclictest0-21swapper/209:39:192
4187500991816,1cyclictest0-21swapper/209:39:192
4187500991816,1cyclictest0-21swapper/209:05:532
4187500991816,1cyclictest0-21swapper/207:30:032
4187500991811,5cyclictest4188667-21ls07:10:212
4187500991810,6cyclictest104937-21df12:25:132
4187495991814,3cyclictest43324-21/usr/sbin/munin09:30:221
4187495991812,5cyclictest83329-21date11:25:011
4187495991812,4cyclictest97595-21cat12:05:141
4187495991812,4cyclictest30952-21/usr/sbin/munin08:55:001
4187495991812,4cyclictest18099-21cat08:15:221
4187495991811,5cyclictest12884-21cat08:00:221
4187495991810,6cyclictest65136-21ntpq10:30:171
4187495991810,6cyclictest65136-21ntpq10:30:171
418749599180,1cyclictest0-21swapper/111:17:541
4187490991816,1cyclictest0-21swapper/012:01:530
4187490991816,1cyclictest0-21swapper/008:44:020
4187490991813,3cyclictest11609-21cron08:00:000
4187490991811,6cyclictest94275-21df11:55:130
4187490991811,5cyclictest13-21ksoftirqd/007:25:210
418750499176,8cyclictest42-21ksoftirqd/311:50:223
4187504991715,1cyclictest3056-21PK-Backend11:24:233
4187500991710,6cyclictest7621-21cat07:45:202
4187500991710,5cyclictest3739-21sed07:35:172
4187500991710,5cyclictest36412-21awk09:10:112
418749599179,6cyclictest14018-21fw_packets08:05:171
4187495991712,3cyclictest54131-21sh10:00:111
4187495991712,3cyclictest49405-21rt-features09:45:191
4187495991712,3cyclictest47179-21sh09:40:141
4187495991712,3cyclictest41355-21latency_hist09:25:011
4187495991711,5cyclictest74625-21sed11:00:001
418749099179,6cyclictest106041-21cron12:29:590
418749099172,12cyclictest43350-21apt09:30:120
4187490991715,1cyclictest3056-21PK-Backend11:30:300
4187490991715,1cyclictest0-21swapper/010:58:130
4187490991715,1cyclictest0-21swapper/007:21:220
4187490991711,5cyclictest80876-21gzip11:15:210
4187490991710,6cyclictest58171-21cat10:10:220
4187504991611,3cyclictest76928-21sh11:05:123
418750499160,15cyclictest38886-21ntp_states09:15:213
418750099168,6cyclictest79008-21ntpq11:10:172
4187495991611,3cyclictest68007-21cpuspeed_turbos10:40:121
4187495991611,3cyclictest50659-21sh09:50:131
418749599160,1cyclictest3056-21PK-Backend08:21:041
4187490991614,1cyclictest0-21swapper/012:32:560
4187490991611,4cyclictest49423-21python309:45:200
4187504991511,3cyclictest3056-21PK-Backend08:04:283
418749599159,4cyclictest4190081-21cut07:15:131
418749599158,5cyclictest16828-21/usr/sbin/munin08:15:011
4187495991510,4cyclictest25024-21cat08:35:201
418749099159,4cyclictest20540-21cron08:25:010
418749099157,6cyclictest436-20systemd-journal08:55:000
4187490991513,1cyclictest3056-21PK-Backend08:35:580
418750499140,13cyclictest673-21chrt07:28:003
418749599140,2cyclictest3056-21PK-Backend08:45:181
418749599140,1cyclictest3056-21PK-Backend10:13:251
418749099149,4cyclictest13-21ksoftirqd/008:15:010
418749099128,3cyclictest0-21swapper/009:14:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional