You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-14 - 18:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Fri Aug 14, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31585924020,9sleep23056-21PK-Backend07:10:012
31503522460,15sleep30-21swapper/307:05:243
40389621100,3sleep20-21swapper/211:15:152
40389621100,3sleep20-21swapper/211:15:152
32794921020,4sleep13056-21PK-Backend07:40:201
3147192970,21sleep013-21ksoftirqd/007:05:170
3148492840,17sleep10-21swapper/107:05:221
3682662500,2sleep20-21swapper/209:35:142
3775762410,3sleep20-21swapper/210:00:202
3857882320,5sleep0385794-21cpuspeed_turbos10:25:110
31592499319,12cyclictest82332-21update-manager08:15:351
3334112300,4sleep33056-21PK-Backend07:55:213
3464662290,7sleep0131ksoftirqd/008:35:010
315924992810,7cyclictest324638-21users07:30:261
31592499270,19cyclictest0-21swapper/111:17:511
31592499270,19cyclictest0-21swapper/111:17:501
315924992619,6cyclictest3056-21PK-Backend09:16:451
139099260,4rtkit-daemon1389-21rtkit-daemon12:24:450
3429662250,3chrt342970-21munin-plugin-st08:25:002
139099250,3rtkit-daemon1389-21rtkit-daemon11:24:300
3637042240,8sleep126-21ksoftirqd/109:20:211
31592499240,18cyclictest3056-21PK-Backend12:18:481
4322132230,4chrt0-21swapper/212:35:172
315932992321,1cyclictest0-21swapper/309:37:433
315924992321,1cyclictest3056-21PK-Backend11:00:201
315924992316,1cyclictest0-21swapper/107:20:451
4171712220,4sleep30-21swapper/311:55:003
315932992220,1cyclictest0-21swapper/309:54:123
315932992219,2cyclictest3056-21PK-Backend12:30:263
315932992219,2cyclictest3056-21PK-Backend11:13:083
315932992219,2cyclictest3056-21PK-Backend10:39:343
315932992214,6cyclictest332006-21cron07:55:013
31593299220,20cyclictest3056-21PK-Backend12:00:333
31593299220,19cyclictest343232-21apt08:25:103
315927992219,2cyclictest339495-21apt08:15:132
315927992214,6cyclictest388845-21cron10:35:002
315924992220,1cyclictest3056-21PK-Backend08:33:321
315924992219,2cyclictest3056-21PK-Backend10:34:251
315924992215,5cyclictest334399-21grep08:00:131
31592499220,21cyclictest3056-21PK-Backend10:20:521
315922992220,1cyclictest0-21swapper/012:01:110
315922992220,1cyclictest0-21swapper/010:03:220
315922992212,7cyclictest385120-21apt-get10:25:000
31592299220,21cyclictest0-21swapper/011:45:020
139099220,3rtkit-daemon1389-21rtkit-daemon12:29:291
139099220,19rtkit-daemon410442-21apt11:35:101
139099220,18rtkit-daemon315915-21cyclictest08:53:390
4076902210,4sleep33056-21PK-Backend11:25:183
31593299215,14cyclictest383402-21basename10:20:003
315932992119,1cyclictest3056-21PK-Backend10:44:213
315932992119,1cyclictest3056-21PK-Backend08:42:343
315932992119,1cyclictest0-21swapper/310:34:473
315932992119,1cyclictest0-21swapper/309:24:313
315932992119,1cyclictest0-21swapper/309:13:233
315932992119,1cyclictest0-21swapper/308:35:113
315932992119,1cyclictest0-21swapper/308:07:123
315932992119,1cyclictest0-21swapper/307:45:223
315932992118,2cyclictest3056-21PK-Backend09:29:183
31593299210,20cyclictest0-21swapper/307:42:383
31593299210,19cyclictest3056-21PK-Backend11:34:513
31593299210,19cyclictest3056-21PK-Backend08:03:193
31593299210,19cyclictest0-21swapper/307:15:143
315927992119,1cyclictest0-21swapper/209:19:422
315927992114,5cyclictest436-20systemd-journal09:15:002
315927992113,6cyclictest319297-21threads07:15:222
31592799211,19cyclictest0-21swapper/208:34:392
31592799210,19cyclictest0-21swapper/211:43:122
315924992119,1cyclictest3056-21PK-Backend12:08:541
315924992119,1cyclictest3056-21PK-Backend10:27:551
315924992119,1cyclictest3056-21PK-Backend09:12:171
315924992119,1cyclictest3056-21PK-Backend08:11:011
315924992119,1cyclictest0-21swapper/112:00:081
315924992119,1cyclictest0-21swapper/111:52:321
315924992119,1cyclictest0-21swapper/110:39:451
315924992119,1cyclictest0-21swapper/110:06:131
315924992119,1cyclictest0-21swapper/109:46:461
315924992119,1cyclictest0-21swapper/109:25:091
315924992119,1cyclictest0-21swapper/108:39:331
315924992119,1cyclictest0-21swapper/107:27:411
315924992119,1cyclictest0-21swapper/107:11:041
315924992118,2cyclictest3056-21PK-Backend08:43:381
315924992113,6cyclictest319830-21grep07:19:591
31592499210,20cyclictest0-21swapper/108:57:181
31592499210,20cyclictest0-21swapper/107:55:471
31592499210,19cyclictest3056-21PK-Backend08:20:531
315922992119,1cyclictest0-21swapper/012:30:240
315922992119,1cyclictest0-21swapper/011:09:410
315922992119,1cyclictest0-21swapper/009:28:490
315922992119,1cyclictest0-21swapper/009:17:120
315922992119,1cyclictest0-21swapper/008:42:320
315922992119,1cyclictest0-21swapper/008:39:480
315922992119,1cyclictest0-21swapper/007:36:140
315922992119,1cyclictest0-21swapper/007:30:260
315922992119,1cyclictest0-21swapper/007:21:330
315922992118,2cyclictest3056-21PK-Backend10:45:410
315922992114,5cyclictest883-21dbus-daemon08:45:350
3774762200,4chrt0-21swapper/110:00:181
315932992018,1cyclictest3056-21PK-Backend12:21:083
315932992018,1cyclictest3056-21PK-Backend07:32:443
315932992018,1cyclictest0-21swapper/312:09:173
315932992018,1cyclictest0-21swapper/309:47:293
315932992018,1cyclictest0-21swapper/308:22:203
315932992018,1cyclictest0-21swapper/308:15:443
31593299201,6cyclictest344940-21/usr/sbin/munin08:30:223
315932992015,3cyclictest370254-21irqstats09:40:153
315932992015,3cyclictest361579-21irqstats09:15:143
315932992013,5cyclictest399889-21wc11:05:003
315932992011,7cyclictest376065-21users09:55:243
31593299201,16cyclictest0-21swapper/308:45:213
31593299200,19cyclictest3056-21PK-Backend07:26:563
31593299200,19cyclictest3056-21PK-Backend07:22:313
31593299200,19cyclictest0-21swapper/309:09:313
31593299200,19cyclictest0-21swapper/307:37:593
31593299200,18cyclictest381646-21cron10:14:593
31593299200,18cyclictest381646-21cron10:14:593
315927992018,1cyclictest0-21swapper/212:19:142
315927992018,1cyclictest0-21swapper/211:28:182
315927992018,1cyclictest0-21swapper/211:06:552
315927992018,1cyclictest0-21swapper/208:27:162
315927992013,5cyclictest394456-21cut10:50:012
315927992012,6cyclictest406708-21munin-run11:24:592
31592799200,19cyclictest0-21swapper/211:13:242
31592799200,19cyclictest0-21swapper/210:23:232
31592799200,19cyclictest0-21swapper/210:12:412
31592799200,19cyclictest0-21swapper/210:12:402
31592799200,19cyclictest0-21swapper/209:50:092
31592799200,19cyclictest0-21swapper/209:47:402
31592799200,19cyclictest0-21swapper/209:40:582
31592799200,19cyclictest0-21swapper/209:34:482
31592799200,19cyclictest0-21swapper/208:45:352
31592799200,19cyclictest0-21swapper/208:09:112
31592799200,19cyclictest0-21swapper/207:56:272
31592799200,19cyclictest0-21swapper/207:54:312
31592799200,19cyclictest0-21swapper/207:27:552
315924992018,1cyclictest3056-21PK-Backend10:14:581
315924992018,1cyclictest3056-21PK-Backend10:14:581
315924992018,1cyclictest3056-21PK-Backend09:43:081
315924992018,1cyclictest3056-21PK-Backend09:02:181
315924992018,1cyclictest3056-21PK-Backend08:49:121
315924992018,1cyclictest3056-21PK-Backend08:25:341
315924992018,1cyclictest3056-21PK-Backend07:52:411
315924992018,1cyclictest0-21swapper/112:33:281
315924992018,1cyclictest0-21swapper/111:24:201
315924992012,6cyclictest418845-21/usr/sbin/munin12:00:001
31592499200,1cyclictest3056-21PK-Backend09:07:521
31592499200,19cyclictest3056-21PK-Backend09:54:371
31592499200,19cyclictest3056-21PK-Backend07:49:241
31592499200,19cyclictest3056-21PK-Backend07:36:471
31592499200,19cyclictest0-21swapper/111:30:141
31592299202,16cyclictest357359-21cron09:05:000
315922992018,1cyclictest0-21swapper/011:34:010
315922992018,1cyclictest0-21swapper/011:19:330
315922992018,1cyclictest0-21swapper/011:19:330
315922992018,1cyclictest0-21swapper/010:57:570
315922992018,1cyclictest0-21swapper/010:16:270
315922992018,1cyclictest0-21swapper/010:14:110
315922992018,1cyclictest0-21swapper/010:14:100
315922992018,1cyclictest0-21swapper/009:34:090
315922992018,1cyclictest0-21swapper/007:57:020
315922992018,1cyclictest0-21swapper/007:54:320
31592299200,19cyclictest0-21swapper/008:59:000
31592299200,19cyclictest0-21swapper/008:26:150
139099200,3rtkit-daemon1389-21rtkit-daemon11:27:441
31593299197,10cyclictest42-21ksoftirqd/312:10:063
31593299197,10cyclictest42-21ksoftirqd/312:10:063
315932991917,1cyclictest3056-21PK-Backend11:59:593
315932991912,5cyclictest356272-21df09:00:133
315932991912,5cyclictest339210-21cron08:15:003
315932991911,6cyclictest393514-21grep10:45:193
31593299191,15cyclictest367807-21unin-run09:35:013
31593299190,1cyclictest3056-21PK-Backend11:20:103
31593299190,1cyclictest3056-21PK-Backend10:05:473
31593299190,1cyclictest0-21swapper/310:50:153
31593299190,1cyclictest0-21swapper/308:57:343
31593299190,18cyclictest0-21swapper/307:14:403
315927991917,1cyclictest0-21swapper/212:05:432
315927991917,1cyclictest0-21swapper/211:49:412
315927991917,1cyclictest0-21swapper/210:17:272
315927991917,1cyclictest0-21swapper/210:05:382
315927991917,1cyclictest0-21swapper/209:26:572
315927991917,1cyclictest0-21swapper/209:00:002
315927991917,1cyclictest0-21swapper/208:58:052
315927991917,1cyclictest0-21swapper/208:42:172
315927991915,3cyclictest978-21snmpd09:08:462
315927991912,5cyclictest335962-21sed08:05:012
31592799190,1cyclictest0-21swapper/212:30:312
31592799190,1cyclictest0-21swapper/212:00:562
31592799190,1cyclictest0-21swapper/208:51:092
31592799190,1cyclictest0-21swapper/207:43:492
31592799190,1cyclictest0-21swapper/207:14:162
31592799190,18cyclictest0-21swapper/211:01:002
315924991917,1cyclictest0-21swapper/110:43:231
315924991917,1cyclictest0-21swapper/109:38:541
315924991917,1cyclictest0-21swapper/108:51:441
315924991916,2cyclictest3056-21PK-Backend10:49:171
31592499190,1cyclictest3056-21PK-Backend08:09:391
31592299192,13cyclictest427773-21apt12:25:110
315922991917,1cyclictest3056-21PK-Backend08:16:500
315922991917,1cyclictest0-21swapper/011:41:370
315922991913,4cyclictest422296-21grep12:09:590
315922991913,4cyclictest399781-21cut11:05:000
315922991912,5cyclictest407416-21cat11:25:150
315922991911,6cyclictest335367-21awk08:00:220
315922991910,7cyclictest392321-21apt-get10:45:010
31592299190,1cyclictest0-21swapper/010:54:320
31592299190,18cyclictest0-21swapper/010:06:120
31592299190,18cyclictest0-21swapper/007:13:360
139099190,16rtkit-daemon0-21swapper/011:10:570
31593299181,5cyclictest377339-21wc10:00:173
315932991813,3cyclictest400215-21grep11:05:123
315932991812,3cyclictest425774-21turbostat.cron12:20:003
315932991811,5cyclictest385437-21date10:25:013
315927991816,1cyclictest0-21swapper/212:11:382
315927991816,1cyclictest0-21swapper/212:11:372
315927991816,1cyclictest0-21swapper/210:53:582
315927991816,1cyclictest0-21swapper/209:59:442
315927991816,1cyclictest0-21swapper/207:24:032
315927991814,3cyclictest391024-21cpuspeed_turbos10:40:122
315927991813,3cyclictest326234-21threads07:35:222
315927991812,4cyclictest338758-21cat08:10:222
315927991810,7cyclictest330316-21unin-run07:50:012
315924991816,1cyclictest0-21swapper/112:39:571
315924991816,1cyclictest0-21swapper/111:44:541
315922991816,1cyclictest0-21swapper/009:55:050
315922991815,2cyclictest3056-21PK-Backend09:50:320
31592299181,12cyclictest1095-21runrttasks10:35:210
31592299180,1cyclictest3056-21PK-Backend11:54:420
31592299180,1cyclictest0-21swapper/007:29:030
315932991714,2cyclictest3056-21PK-Backend08:51:253
315932991711,5cyclictest42-21ksoftirqd/310:55:013
315932991711,5cyclictest42-21ksoftirqd/310:25:013
315927991715,1cyclictest0-21swapper/211:36:442
315927991715,1cyclictest0-21swapper/209:22:462
315927991710,5cyclictest416080-21awk11:50:152
315924991711,5cyclictest401630-21sed11:10:011
315924991710,5cyclictest414612-21ntpq11:45:201
31592299172,13cyclictest432977-21ls12:40:000
315922991715,1cyclictest3056-21PK-Backend08:05:070
315922991715,1cyclictest0-21swapper/009:14:080
315922991715,1cyclictest0-21swapper/007:17:550
315922991711,5cyclictest13-21ksoftirqd/009:45:370
315922991710,5cyclictest13-21ksoftirqd/011:55:180
315932991614,1cyclictest3056-21PK-Backend11:39:373
315932991614,1cyclictest3056-21PK-Backend11:19:373
315932991614,1cyclictest3056-21PK-Backend11:19:363
315927991614,1cyclictest0-21swapper/210:39:372
315927991611,3cyclictest386195-21latency10:25:172
315927991610,5cyclictest410210-21grep11:35:002
315927991610,4cyclictest426747-21tr12:20:182
315924991611,4cyclictest376241-21seq09:57:281
31592499160,14cyclictest366980-21grep09:30:181
31592299169,5cyclictest411163-21ntpq11:35:190
31592299169,5cyclictest364292-21cron09:24:590
31592299169,4cyclictest2156-21gnome-shell08:10:090
315922991614,1cyclictest0-21swapper/008:21:150
315922991614,1cyclictest0-21swapper/007:46:280
315922991613,2cyclictest3056-21PK-Backend09:36:190
315922991612,3cyclictest327056-21/usr/sbin/munin07:40:190
315922991611,3cyclictest371242-21latency_hist09:45:000
31592299160,15cyclictest0-21swapper/009:05:440
315932991513,1cyclictest3056-21PK-Backend12:39:463
31592799159,4cyclictest396424-21apt-key10:55:022
31592799158,5cyclictest347218-21cat08:35:142
315927991513,1cyclictest0-21swapper/207:34:312
31592499159,4cyclictest401912-21grep11:10:111
315924991511,3cyclictest398051-21idleruntime-cro11:00:011
315924991510,3cyclictest426412-21sh12:20:141
31593299140,6cyclictest3056-21PK-Backend12:25:173
31593299140,13cyclictest3056-21PK-Backend11:40:263
315927991411,2cyclictest1095-21runrttasks12:25:142
31592799140,6cyclictest0-21swapper/211:55:172
31592299146,6cyclictest387853-21cat10:30:150
315922991411,2cyclictest422546-21/usr/sbin/munin12:10:160
315922991411,2cyclictest422546-21/usr/sbin/munin12:10:160
31593299139,3cyclictest3056-21PK-Backend11:45:383
31592299132,7cyclictest2156-21gnome-shell12:17:090
31592499128,3cyclictest3056-21PK-Backend10:17:501
31592499117,3cyclictest3056-21PK-Backend12:13:561
31592499117,3cyclictest3056-21PK-Backend12:13:561
3159249996,2cyclictest3945472chrt10:50:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional