You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-19 - 12:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Sat Sep 19, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
362571121050,3sleep13056-21PK-Backend21:05:021
363129921000,4sleep13056-21PK-Backend21:20:011
139099900,58rtkit-daemon0-21swapper/019:06:210
358452027540,29sleep20-21swapper/219:08:472
358317127246,16sleep10-21swapper/119:05:121
358441026240,14sleep30-21swapper/319:07:223
36218412590,4sleep23621832-21dump-pmu-power20:55:012
36593782510,4sleep00-21swapper/022:35:220
36761302490,4sleep13056-21PK-Backend23:25:161
36369032360,4sleep23636902-21cpuspeed_turbos21:35:132
36369032360,4sleep23636902-21cpuspeed_turbos21:35:132
36088882290,4sleep13608892-21needreboot20:15:171
3584766992320,2cyclictest3056-21PK-Backend22:31:391
3584766992320,2cyclictest3056-21PK-Backend22:31:391
3584761992321,1cyclictest0-21swapper/000:29:170
36851472220,4chrt0-21swapper/023:50:170
3584772992220,1cyclictest0-21swapper/320:02:083
3584772992215,4cyclictest3630411-21python321:15:203
3584772992214,6cyclictest436-20systemd-journal20:30:003
3584772992214,6cyclictest3594097-21cron19:35:003
358477299221,18cyclictest36424702sleep321:50:103
3584770992220,1cyclictest0-21swapper/223:20:382
3584770992220,1cyclictest0-21swapper/222:05:092
3584770992220,1cyclictest0-21swapper/220:15:102
3584770992220,1cyclictest0-21swapper/200:31:362
358477099220,21cyclictest0-21swapper/220:42:342
358477099220,20cyclictest0-21swapper/221:08:102
3584766992219,2cyclictest3056-21PK-Backend23:35:411
3584766992219,2cyclictest3056-21PK-Backend22:20:171
3584766992219,2cyclictest3056-21PK-Backend00:30:441
3584766992219,2cyclictest3056-21PK-Backend00:07:591
358476699220,20cyclictest3056-21PK-Backend20:37:361
3584761992220,1cyclictest0-21swapper/023:41:380
3584761992220,1cyclictest0-21swapper/023:22:050
3584761992220,1cyclictest0-21swapper/022:06:210
3584761992220,1cyclictest0-21swapper/019:25:000
139099220,3rtkit-daemon0-21swapper/019:59:370
139099220,14rtkit-daemon3056-21PK-Backend23:57:161
36954852210,5chrt0-21swapper/300:20:153
358477299219,10cyclictest3601818-21cat19:55:163
358477299212,16cyclictest3634749-21latency_hist21:30:003
3584772992119,1cyclictest0-21swapper/323:47:313
3584772992119,1cyclictest0-21swapper/321:38:423
3584772992119,1cyclictest0-21swapper/321:38:413
3584772992119,1cyclictest0-21swapper/321:30:233
3584772992119,1cyclictest0-21swapper/320:43:323
3584772992119,1cyclictest0-21swapper/320:25:003
3584772992119,1cyclictest0-21swapper/319:47:363
3584772992119,1cyclictest0-21swapper/319:26:273
3584772992119,1cyclictest0-21swapper/319:20:523
3584772992119,1cyclictest0-21swapper/300:16:523
3584772992112,6cyclictest3645655-21cron22:00:013
358477299211,17cyclictest3664104-21memory22:50:183
358477299210,20cyclictest0-21swapper/323:53:343
358477299210,20cyclictest0-21swapper/323:36:023
358477299210,20cyclictest0-21swapper/323:21:233
358477299210,20cyclictest0-21swapper/321:11:313
358477299210,20cyclictest0-21swapper/319:44:373
358477299210,20cyclictest0-21swapper/319:17:003
358477299210,20cyclictest0-21swapper/300:30:373
358477299210,1cyclictest0-21swapper/323:10:273
358477299210,19cyclictest3056-21PK-Backend20:15:553
358477299210,19cyclictest0-21swapper/322:11:193
358477099217,12cyclictest3599332-21unin-run19:50:002
358477099212,5cyclictest886-21NetworkManager23:50:172
3584770992119,1cyclictest0-21swapper/223:07:472
3584770992119,1cyclictest0-21swapper/219:58:142
3584770992119,1cyclictest0-21swapper/219:51:102
3584770992119,1cyclictest0-21swapper/219:20:152
3584770992119,1cyclictest0-21swapper/200:35:142
3584770992119,1cyclictest0-21swapper/200:12:022
3584770992119,1cyclictest0-21swapper/200:08:582
3584770992119,1cyclictest0-21swapper/200:01:142
3584770992113,6cyclictest3661843-21awk22:45:102
358477099210,8cyclictest36793932grep23:35:132
358477099210,20cyclictest0-21swapper/223:57:032
358477099210,20cyclictest0-21swapper/222:11:502
358477099210,20cyclictest0-21swapper/221:59:402
358477099210,20cyclictest0-21swapper/221:42:042
358477099210,20cyclictest0-21swapper/221:42:042
358477099210,20cyclictest0-21swapper/221:30:542
358477099210,20cyclictest0-21swapper/220:06:522
3584766992119,1cyclictest3056-21PK-Backend23:41:491
3584766992119,1cyclictest3056-21PK-Backend19:51:591
3584766992119,1cyclictest0-21swapper/123:13:161
3584766992118,2cyclictest3056-21PK-Backend23:20:191
3584766992118,2cyclictest3056-21PK-Backend00:16:361
358476699210,20cyclictest3056-21PK-Backend23:15:121
358476699210,20cyclictest3056-21PK-Backend22:54:031
358476699210,1cyclictest3056-21PK-Backend20:54:391
358476699210,1cyclictest3056-21PK-Backend20:12:341
358476699210,19cyclictest3056-21PK-Backend23:08:241
358476699210,19cyclictest3056-21PK-Backend22:46:121
358476699210,19cyclictest3056-21PK-Backend22:35:321
358476699210,19cyclictest3056-21PK-Backend19:42:141
358476699210,19cyclictest3056-21PK-Backend19:15:571
358476699210,19cyclictest3056-21PK-Backend00:36:301
358476699210,19cyclictest0-21swapper/120:20:051
3584761992119,1cyclictest0-21swapper/023:57:200
3584761992119,1cyclictest0-21swapper/023:34:350
3584761992119,1cyclictest0-21swapper/023:26:100
3584761992119,1cyclictest0-21swapper/022:47:520
3584761992119,1cyclictest0-21swapper/021:54:240
3584761992119,1cyclictest0-21swapper/021:44:440
3584761992119,1cyclictest0-21swapper/021:44:440
3584761992119,1cyclictest0-21swapper/020:57:530
3584761992119,1cyclictest0-21swapper/020:30:020
3584761992119,1cyclictest0-21swapper/020:27:180
3584761992119,1cyclictest0-21swapper/020:10:020
3584761992119,1cyclictest0-21swapper/019:51:020
3584761992119,1cyclictest0-21swapper/019:36:140
3584761992119,1cyclictest0-21swapper/000:07:270
358476199210,8cyclictest3627294-21awk21:10:000
358476199210,8cyclictest36167111latency_hist20:40:000
358476199210,20cyclictest0-21swapper/020:00:140
358476199210,20cyclictest0-21swapper/000:24:240
358476199210,20cyclictest0-21swapper/000:16:060
139099210,15rtkit-daemon3056-21PK-Backend20:55:091
139099210,14rtkit-daemon3056-21PK-Backend23:49:141
139099210,14rtkit-daemon3056-21PK-Backend20:00:311
139099210,14rtkit-daemon0-21swapper/022:29:140
139099210,13rtkit-daemon0-21swapper/021:22:460
36434042200,4sleep13056-21PK-Backend21:50:211
358477299203,4cyclictest3658884-21cat22:35:173
3584772992018,1cyclictest0-21swapper/323:57:113
3584772992018,1cyclictest0-21swapper/323:16:363
3584772992018,1cyclictest0-21swapper/322:44:383
3584772992018,1cyclictest0-21swapper/322:33:583
3584772992018,1cyclictest0-21swapper/322:33:573
3584772992018,1cyclictest0-21swapper/319:54:253
3584772992018,1cyclictest0-21swapper/300:37:543
3584772992016,2cyclictest36522202sleep322:15:243
3584772992015,3cyclictest3615532-21df20:35:133
3584772992014,4cyclictest2156-21gnome-shell00:03:093
358477299201,10cyclictest3652958-21/usr/sbin/munin22:20:113
358477299200,1cyclictest0-21swapper/300:07:183
358477299200,19cyclictest3056-21PK-Backend21:09:543
358477299200,19cyclictest0-21swapper/323:30:283
358477299200,19cyclictest0-21swapper/323:05:333
358477299200,19cyclictest0-21swapper/320:12:363
358477299200,19cyclictest0-21swapper/300:11:383
3584770992018,1cyclictest0-21swapper/223:47:422
3584770992018,1cyclictest0-21swapper/223:15:522
3584770992018,1cyclictest0-21swapper/219:41:512
3584770992018,1cyclictest0-21swapper/219:13:532
3584770992018,1cyclictest0-21swapper/200:23:042
358477099201,6cyclictest3610344-21entropy20:20:152
3584770992015,3cyclictest3694827-21sed00:20:002
358477099200,1cyclictest0-21swapper/221:45:072
358477099200,1cyclictest0-21swapper/219:16:232
358477099200,19cyclictest0-21swapper/222:51:522
358477099200,19cyclictest0-21swapper/222:39:012
358477099200,19cyclictest0-21swapper/222:33:212
358477099200,19cyclictest0-21swapper/222:33:202
358477099200,19cyclictest0-21swapper/221:25:002
358477099200,19cyclictest0-21swapper/221:10:122
358477099200,19cyclictest0-21swapper/219:25:362
358477099200,19cyclictest0-21swapper/200:28:522
3584766992018,1cyclictest3056-21PK-Backend23:54:531
3584766992018,1cyclictest3056-21PK-Backend22:25:511
3584766992018,1cyclictest3056-21PK-Backend21:35:021
3584766992018,1cyclictest3056-21PK-Backend21:35:011
3584766992018,1cyclictest3056-21PK-Backend19:59:301
3584766992018,1cyclictest3056-21PK-Backend00:28:531
3584766992018,1cyclictest3056-21PK-Backend00:21:501
3584766992018,1cyclictest3056-21PK-Backend00:02:031
358476699200,19cyclictest3056-21PK-Backend19:48:351
358476699200,19cyclictest3056-21PK-Backend19:26:311
3584761992018,1cyclictest3056-21PK-Backend21:30:590
3584761992018,1cyclictest0-21swapper/023:11:100
3584761992018,1cyclictest0-21swapper/023:07:180
3584761992018,1cyclictest0-21swapper/021:55:400
3584761992018,1cyclictest0-21swapper/021:25:450
3584761992018,1cyclictest0-21swapper/019:49:200
3584761992018,1cyclictest0-21swapper/019:41:280
3584761992018,1cyclictest0-21swapper/019:15:060
3584761992018,1cyclictest0-21swapper/000:04:020
3584761992014,3cyclictest13-21ksoftirqd/022:30:110
3584761992014,3cyclictest13-21ksoftirqd/022:30:110
358476199200,1cyclictest0-21swapper/000:13:020
358476199200,19cyclictest0-21swapper/023:45:590
358476199200,19cyclictest0-21swapper/023:17:190
358476199200,19cyclictest0-21swapper/022:43:450
358476199200,19cyclictest0-21swapper/021:18:150
358476199200,19cyclictest0-21swapper/020:17:050
358476199200,19cyclictest0-21swapper/019:33:580
358476199200,19cyclictest0-21swapper/000:30:120
139099200,14rtkit-daemon3056-21PK-Backend22:01:461
3584772991917,1cyclictest0-21swapper/322:27:363
3584772991917,1cyclictest0-21swapper/322:04:373
358477299191,5cyclictest3647668-21apt22:05:123
358477299191,5cyclictest3625300-21munin-run21:05:003
3584772991912,5cyclictest3675707-21awk23:25:123
3584772991912,5cyclictest3666195-21threads22:55:213
3584772991911,6cyclictest3638205-21apt-get21:40:043
3584772991911,6cyclictest3638205-21apt-get21:40:033
358477299190,1cyclictest0-21swapper/320:06:553
358477299190,17cyclictest3632285-21ntp_states21:20:183
358477299190,16cyclictest3696729-21apt00:25:123
358477099191,7cyclictest34-21ksoftirqd/223:00:152
3584770991917,1cyclictest0-21swapper/223:26:002
3584770991917,1cyclictest0-21swapper/223:14:162
3584770991917,1cyclictest0-21swapper/221:52:442
3584770991917,1cyclictest0-21swapper/220:35:592
3584770991917,1cyclictest0-21swapper/220:04:302
3584770991915,3cyclictest978-21snmpd20:56:322
3584770991911,6cyclictest3625284-21apt-get21:05:002
358477099190,1cyclictest0-21swapper/223:41:062
358477099190,1cyclictest0-21swapper/222:27:052
358477099190,1cyclictest0-21swapper/222:20:152
358477099190,1cyclictest0-21swapper/221:27:512
358477099190,1cyclictest0-21swapper/220:30:192
358477099190,1cyclictest0-21swapper/220:12:202
358477099190,1cyclictest0-21swapper/219:31:522
358477099190,18cyclictest0-21swapper/223:33:092
3584766991917,1cyclictest3056-21PK-Backend22:08:131
3584766991917,1cyclictest3056-21PK-Backend20:09:181
3584766991917,1cyclictest3056-21PK-Backend19:23:001
3584766991916,2cyclictest3056-21PK-Backend21:34:481
358476699190,1cyclictest3056-21PK-Backend20:26:061
358476699190,1cyclictest3056-21PK-Backend19:13:271
358476699190,18cyclictest3056-21PK-Backend21:13:111
358476699190,18cyclictest3056-21PK-Backend20:42:421
3584761991917,1cyclictest0-21swapper/023:35:300
3584761991917,1cyclictest0-21swapper/022:15:060
3584761991917,1cyclictest0-21swapper/020:21:160
3584761991912,6cyclictest3641931-21apt-get21:50:000
358476199190,1cyclictest0-21swapper/022:50:280
358476199190,1cyclictest0-21swapper/022:00:050
358476199190,1cyclictest0-21swapper/020:44:290
3584772991816,1cyclictest0-21swapper/323:40:553
3584772991816,1cyclictest0-21swapper/322:45:123
3584772991813,3cyclictest3640458-21acpi21:45:113
3584772991813,3cyclictest3614928-21munin-run20:35:003
3584772991812,4cyclictest3585329-21diskstats19:10:133
3584770991811,5cyclictest3660508-21cat22:40:182
3584770991810,6cyclictest3594367-21awk19:35:122
358477099180,1cyclictest0-21swapper/222:04:072
3584766991816,1cyclictest3056-21PK-Backend22:40:181
3584766991816,1cyclictest3056-21PK-Backend21:59:211
3584766991816,1cyclictest3056-21PK-Backend19:35:311
3584766991815,2cyclictest3056-21PK-Backend21:29:341
3584761991816,1cyclictest0-21swapper/021:36:270
3584761991816,1cyclictest0-21swapper/021:36:260
3584761991816,1cyclictest0-21swapper/021:00:100
3584761991816,1cyclictest0-21swapper/019:12:350
3584761991812,5cyclictest3649499-21sort22:10:130
3584761991811,4cyclictest2156-21gnome-shell20:54:090
358476199180,8cyclictest3627707-21cpuspeed_turbos21:10:120
358477299179,5cyclictest0-21swapper/320:55:123
3584772991712,3cyclictest3667148-21grep23:00:113
3584772991710,5cyclictest1-21systemd20:50:013
3584772991710,5cyclictest1-21systemd20:50:013
358477299170,15cyclictest3594376-21apt19:35:133
358477099179,5cyclictest3651992-21python322:15:212
3584770991712,4cyclictest3666710-21cron23:00:012
3584770991711,4cyclictest436-20systemd-journal21:17:402
3584766991715,1cyclictest3056-21PK-Backend21:43:121
3584766991715,1cyclictest3056-21PK-Backend21:43:121
3584766991715,1cyclictest3056-21PK-Backend00:12:171
3584766991714,2cyclictest3056-21PK-Backend23:33:311
3584766991714,2cyclictest3056-21PK-Backend22:55:111
3584761991715,1cyclictest0-21swapper/022:55:210
3584761991715,1cyclictest0-21swapper/022:22:300
3584761991713,3cyclictest3586661-21munin-node20:45:120
3584761991713,3cyclictest3586661-21munin-node20:45:110
3584761991712,3cyclictest3590882-21acpi19:25:110
3584761991710,5cyclictest3668494-21latency_hist23:05:010
358477299161,11cyclictest3620365-21apt20:50:113
3584770991614,1cyclictest0-21swapper/220:27:352
3584766991614,1cyclictest3056-21PK-Backend20:31:131
3584761991610,5cyclictest3701653-21cron00:40:000
358477099158,5cyclictest3618859-21expr20:45:132
358477099158,5cyclictest3618859-21expr20:45:132
3584766991513,1cyclictest3056-21PK-Backend19:33:001
3584761991513,1cyclictest0-21swapper/020:07:100
3584766991412,1cyclictest3056-21PK-Backend21:45:161
3584766991411,2cyclictest3056-21PK-Backend21:24:331
358476699140,6cyclictest3056-21PK-Backend22:10:201
358476699139,3cyclictest3056-21PK-Backend23:03:341
358476699139,3cyclictest3056-21PK-Backend21:02:531
358476699139,3cyclictest3056-21PK-Backend20:49:051
358476699139,3cyclictest3056-21PK-Backend20:49:051
3584766991210,1cyclictest3056-21PK-Backend22:15:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional