You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-20 - 10:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu Aug 20, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
34261392940,7sleep20-21swapper/219:05:272
34261392940,7sleep20-21swapper/219:05:272
342571028541,12sleep00-21swapper/019:05:180
342571028541,12sleep00-21swapper/019:05:180
342667127943,14sleep10-21swapper/119:08:321
342667127943,14sleep10-21swapper/119:08:321
139099740,61rtkit-daemon0-21swapper/319:06:053
139099740,61rtkit-daemon0-21swapper/319:06:043
34405082550,2sleep30-21swapper/319:45:193
34746562510,4sleep00-21swapper/021:20:230
34271212510,3sleep03056-21PK-Backend19:10:110
34763742500,4sleep13056-21PK-Backend21:25:221
139099350,26rtkit-daemon0-21swapper/221:15:002
342692599340,20cyclictest0-21swapper/321:16:033
342692599310,22cyclictest0-21swapper/321:29:083
342692599310,18cyclictest0-21swapper/321:44:153
342692599310,16cyclictest0-21swapper/321:47:313
342692599300,17cyclictest0-21swapper/300:26:323
342692599300,16cyclictest3452420-21cstates20:20:133
342692599290,17cyclictest0-21swapper/323:16:093
342692599290,15cyclictest3520106-21timerwakeupswit23:30:193
342692599290,13cyclictest3470942-21needreboot21:10:183
342692599281,26cyclictest3504910-21cron22:49:593
3426925992810,15cyclictest3508677-21apt23:00:123
342692599280,26cyclictest3518064-21ntpq23:25:183
342692599280,20cyclictest0-21swapper/320:28:573
342692599280,16cyclictest0-21swapper/323:21:243
342692599280,16cyclictest0-21swapper/322:19:473
342692599280,16cyclictest0-21swapper/322:14:113
342692599280,15cyclictest0-21swapper/300:21:333
35338402260,3sleep3411rcuc/300:10:213
342692599268,16cyclictest3505348-21expr22:50:133
342692599260,7cyclictest0-21swapper/300:36:333
342692599260,13cyclictest0-21swapper/319:20:183
342692599260,12cyclictest0-21swapper/322:05:133
139099260,17rtkit-daemon3473532-21cat21:20:002
342692599259,14cyclictest3527846-21sort23:55:133
342692599258,15cyclictest3500344-21cat22:35:143
342692599257,16cyclictest3535237-21cat00:15:113
3426925992514,9cyclictest3441845-21df19:50:133
3426925992513,10cyclictest3524281-21apt23:45:113
3426925992513,10cyclictest3480008-21users21:35:233
3426925992512,11cyclictest42-21ksoftirqd/319:20:013
3426925992510,14cyclictest3465409-21sed20:55:173
342692599250,23cyclictest0-21swapper/323:05:203
342692599250,22cyclictest3540252-21/usr/sbin/munin00:30:143
342692599250,13cyclictest0-21swapper/320:00:213
3426922992513,10cyclictest3537282-21memory00:20:172
34845652240,4chrt3056-21PK-Backend21:50:161
342692599249,14cyclictest3464048-21sed20:50:213
342692599245,17cyclictest42-21ksoftirqd/319:29:553
342692599241,21cyclictest3457800-21grep20:35:123
3426925992412,10cyclictest3495303-21latency22:20:163
3426925992412,10cyclictest3495303-21latency22:20:163
342692599240,22cyclictest3529781-21cstates00:00:163
342692599240,22cyclictest3436657-21diskstats19:35:133
342692599239,12cyclictest3467553-21open_files21:00:203
342692599237,14cyclictest3526336-21expr23:50:133
342692599232,20cyclictest3447131-21cstates20:05:123
3426925992311,10cyclictest3512114-21/usr/sbin/munin23:10:133
342692599230,10cyclictest3477946-21ntp_states21:30:203
3426922992321,1cyclictest0-21swapper/220:58:542
3426915992321,1cyclictest3056-21PK-Backend20:35:230
3426915992314,7cyclictest3537063-21df00:20:130
139099230,15rtkit-daemon0-21swapper/200:30:172
342692599228,12cyclictest3499728-21latency_hist22:35:013
342692599228,12cyclictest3489174-21date22:05:013
342692599226,14cyclictest3456935-21cat20:30:203
3426925992212,9cyclictest978-21snmpd19:30:213
3426925992212,9cyclictest978-21snmpd19:30:213
342692599221,20cyclictest3487330-21munin-run22:00:013
342692599220,20cyclictest3520792-21acpi23:35:103
342692599220,20cyclictest3501928-21cstates22:40:093
342692599220,20cyclictest3462415-21/usr/sbin/munin20:45:193
342692599220,20cyclictest3448660-21/usr/sbin/munin20:10:153
342692599220,19cyclictest0-21swapper/321:05:133
342692599220,10cyclictest3497265-21ntp_states22:25:183
3426922992215,5cyclictest3436004-21cron19:35:002
3426922992215,5cyclictest3436004-21cron19:34:592
3426922992214,6cyclictest3462797-21cron20:50:002
3426922992212,7cyclictest436-20systemd-journal20:10:002
342691799220,20cyclictest3056-21PK-Backend20:20:131
342691799220,20cyclictest3056-21PK-Backend20:01:271
3426915992220,1cyclictest0-21swapper/023:50:240
3426915992220,1cyclictest0-21swapper/023:24:490
3426915992210,4cyclictest3487609-21wc22:00:120
139099220,14rtkit-daemon0-21swapper/222:50:162
342692599218,11cyclictest3473866-21grep21:20:143
3426925992119,1cyclictest0-21swapper/323:41:313
3426925992113,7cyclictest3438016-21apt19:40:103
342692599210,20cyclictest0-21swapper/320:15:173
342692599210,11cyclictest0-21swapper/319:55:173
3426922992119,1cyclictest0-21swapper/223:30:442
3426922992119,1cyclictest0-21swapper/223:29:552
3426922992119,1cyclictest0-21swapper/222:32:372
3426922992119,1cyclictest0-21swapper/222:25:282
3426922992119,1cyclictest0-21swapper/222:05:062
3426922992119,1cyclictest0-21swapper/221:30:532
3426922992119,1cyclictest0-21swapper/221:08:082
3426922992119,1cyclictest0-21swapper/220:32:122
3426922992119,1cyclictest0-21swapper/200:08:222
3426922992118,2cyclictest3056-21PK-Backend23:41:322
342692299210,20cyclictest3056-21PK-Backend22:37:512
342692299210,20cyclictest0-21swapper/222:12:162
342692299210,20cyclictest0-21swapper/220:41:182
342692299210,20cyclictest0-21swapper/220:11:232
342691799210,20cyclictest3056-21PK-Backend19:42:001
342691799210,20cyclictest3056-21PK-Backend19:19:501
342691799210,20cyclictest0-21swapper/100:35:291
342691799210,19cyclictest3056-21PK-Backend23:58:401
342691799210,19cyclictest3056-21PK-Backend23:51:231
342691799210,19cyclictest3056-21PK-Backend23:28:111
342691799210,19cyclictest3056-21PK-Backend23:13:111
342691799210,19cyclictest3056-21PK-Backend22:14:441
342691799210,19cyclictest3056-21PK-Backend22:00:361
342691799210,19cyclictest3056-21PK-Backend21:33:131
342691799210,19cyclictest3056-21PK-Backend20:59:471
342691799210,19cyclictest3056-21PK-Backend20:42:231
342691799210,19cyclictest3056-21PK-Backend19:54:581
3426915992119,1cyclictest0-21swapper/023:59:160
3426915992119,1cyclictest0-21swapper/023:32:060
3426915992119,1cyclictest0-21swapper/023:15:160
3426915992119,1cyclictest0-21swapper/022:39:460
3426915992119,1cyclictest0-21swapper/022:34:330
3426915992119,1cyclictest0-21swapper/021:31:060
3426915992119,1cyclictest0-21swapper/021:04:090
3426915992119,1cyclictest0-21swapper/020:31:310
3426915992119,1cyclictest0-21swapper/020:18:130
3426915992119,1cyclictest0-21swapper/020:02:170
3426915992119,1cyclictest0-21swapper/019:48:510
3426915992119,1cyclictest0-21swapper/019:27:560
3426915992119,1cyclictest0-21swapper/000:31:140
3426915992118,2cyclictest3056-21PK-Backend00:03:020
342691599210,20cyclictest3056-21PK-Backend23:39:570
342691599210,20cyclictest0-21swapper/021:39:580
342691599210,20cyclictest0-21swapper/021:07:130
342691599210,20cyclictest0-21swapper/000:37:090
342691599210,20cyclictest0-21swapper/000:14:020
139099210,15rtkit-daemon3056-21PK-Backend21:37:072
139099210,14rtkit-daemon0-21swapper/200:27:262
139099210,13rtkit-daemon0-21swapper/222:20:222
139099210,13rtkit-daemon0-21swapper/222:20:212
139099210,13rtkit-daemon0-21swapper/220:50:152
3426925992018,1cyclictest0-21swapper/300:07:343
342692599201,7cyclictest3459985-21diskstats20:40:133
342692599201,18cyclictest3427865-21ntp_states19:10:193
342692599200,18cyclictest3508428-21awk23:00:013
342692599200,18cyclictest3484298-21cpuspeed_turbos21:50:133
342692299202,13cyclictest0-21swapper/223:00:122
3426922992018,1cyclictest0-21swapper/223:37:132
3426922992018,1cyclictest0-21swapper/223:15:162
3426922992018,1cyclictest0-21swapper/222:46:232
3426922992018,1cyclictest0-21swapper/221:46:552
3426922992018,1cyclictest0-21swapper/221:41:072
3426922992018,1cyclictest0-21swapper/220:16:242
3426922992018,1cyclictest0-21swapper/219:50:552
3426922992018,1cyclictest0-21swapper/219:19:122
3426922992018,1cyclictest0-21swapper/200:36:482
3426922992018,1cyclictest0-21swapper/200:04:372
3426922992013,5cyclictest3486472-21cat21:55:192
3426922992012,6cyclictest3440485-21ntp_states19:45:182
342692299201,13cyclictest3532975-21apt00:10:122
342692299200,19cyclictest3056-21PK-Backend21:50:462
342692299200,19cyclictest0-21swapper/223:54:092
342692299200,19cyclictest0-21swapper/222:57:182
342692299200,19cyclictest0-21swapper/222:01:482
342692299200,19cyclictest0-21swapper/221:02:342
342692299200,19cyclictest0-21swapper/220:38:002
3426917992018,1cyclictest3056-21PK-Backend20:36:001
3426917992018,1cyclictest3056-21PK-Backend20:25:131
3426917992015,3cyclictest3498409-21tr22:30:121
3426917992013,5cyclictest3483128-21cat21:45:161
3426917992011,7cyclictest3530578-21users00:00:281
342691799200,1cyclictest3056-21PK-Backend23:38:111
342691799200,1cyclictest3056-21PK-Backend21:35:081
342691799200,1cyclictest3056-21PK-Backend21:12:381
342691799200,19cyclictest3056-21PK-Backend23:42:311
342691799200,19cyclictest3056-21PK-Backend22:48:031
342691799200,19cyclictest3056-21PK-Backend20:49:401
342691799200,19cyclictest3056-21PK-Backend20:32:091
342691799200,19cyclictest0-21swapper/122:21:531
342691799200,19cyclictest0-21swapper/122:21:531
342691799200,19cyclictest0-21swapper/121:06:291
342691799200,19cyclictest0-21swapper/119:25:241
342691799200,18cyclictest3056-21PK-Backend23:23:181
342691799200,18cyclictest3056-21PK-Backend23:15:541
342691799200,18cyclictest3056-21PK-Backend22:27:071
342691799200,18cyclictest3056-21PK-Backend19:36:461
3426915992018,1cyclictest0-21swapper/023:46:380
3426915992018,1cyclictest0-21swapper/022:47:450
3426915992018,1cyclictest0-21swapper/022:08:580
3426915992018,1cyclictest0-21swapper/021:57:560
3426915992018,1cyclictest0-21swapper/019:35:410
3426915992018,1cyclictest0-21swapper/019:22:570
3426915992018,1cyclictest0-21swapper/000:06:470
342691599200,1cyclictest0-21swapper/020:55:100
342691599200,1cyclictest0-21swapper/019:32:230
342691599200,1cyclictest0-21swapper/019:32:230
342691599200,19cyclictest0-21swapper/023:29:080
342691599200,19cyclictest0-21swapper/020:23:130
342691599200,19cyclictest0-21swapper/020:13:120
139099200,13rtkit-daemon0-21swapper/223:55:112
34855832190,3sleep03485717-21latency_hist21:55:010
3426922991917,1cyclictest0-21swapper/219:21:422
3426922991911,6cyclictest3516095-21uname23:20:142
3426922991910,7cyclictest883-21dbus-daemon23:50:012
3426922991910,7cyclictest3453941-21awk20:25:122
342692299190,1cyclictest0-21swapper/221:27:002
342692299190,1cyclictest0-21swapper/221:20:042
342692299190,18cyclictest0-21swapper/223:09:012
342692299190,17cyclictest3443935-21ntpq19:55:162
3426917991917,1cyclictest3056-21PK-Backend23:45:281
3426917991917,1cyclictest3056-21PK-Backend21:58:411
3426917991917,1cyclictest3056-21PK-Backend20:11:001
3426917991917,1cyclictest3056-21PK-Backend20:06:471
3426917991917,1cyclictest3056-21PK-Backend19:13:141
3426917991916,2cyclictest3056-21PK-Backend21:20:091
342691799190,1cyclictest3056-21PK-Backend23:07:431
342691799190,1cyclictest3056-21PK-Backend21:42:051
342691799190,1cyclictest3056-21PK-Backend00:21:311
342691799190,18cyclictest0-21swapper/122:37:351
3426915991917,1cyclictest0-21swapper/020:05:560
3426915991917,1cyclictest0-21swapper/019:55:140
3426915991917,1cyclictest0-21swapper/019:42:510
3426915991917,1cyclictest0-21swapper/000:17:480
342691599190,1cyclictest0-21swapper/023:06:160
342691599190,1cyclictest0-21swapper/022:55:560
342691599190,1cyclictest0-21swapper/021:17:470
342691599190,1cyclictest0-21swapper/020:27:390
342691599190,18cyclictest0-21swapper/019:51:360
3426922991816,1cyclictest0-21swapper/220:04:412
3426922991816,1cyclictest0-21swapper/219:41:362
3426922991816,1cyclictest0-21swapper/219:29:122
3426922991816,1cyclictest0-21swapper/200:18:022
3426922991811,5cyclictest436-20systemd-journal23:12:412
3426917991816,1cyclictest3056-21PK-Backend00:30:221
3426917991816,1cyclictest3056-21PK-Backend00:25:161
3426917991816,1cyclictest3056-21PK-Backend00:18:341
3426917991816,1cyclictest0-21swapper/122:18:011
3426917991816,1cyclictest0-21swapper/120:54:271
3426917991815,2cyclictest3056-21PK-Backend21:17:381
342691799180,1cyclictest3056-21PK-Backend22:08:561
3426915991816,1cyclictest0-21swapper/023:40:430
3426915991816,1cyclictest0-21swapper/022:27:240
3426915991816,1cyclictest0-21swapper/021:47:420
3426915991816,1cyclictest0-21swapper/021:26:400
3426915991812,5cyclictest3429631-21cat19:15:130
342691599180,1cyclictest0-21swapper/022:17:300
342691599180,17cyclictest0-21swapper/022:12:150
35059192170,3sleep13056-21PK-Backend22:50:191
342692299173,8cyclictest2156-21gnome-shell22:40:092
3426922991715,1cyclictest0-21swapper/219:38:182
3426922991715,1cyclictest0-21swapper/219:11:072
3426922991711,5cyclictest3453672-21cat20:25:002
3426917991715,1cyclictest3056-21PK-Backend00:13:271
3426917991714,2cyclictest3056-21PK-Backend22:56:131
3426915991715,1cyclictest0-21swapper/023:11:090
3426915991715,1cyclictest0-21swapper/021:41:190
3426915991710,5cyclictest3461064-21cron20:45:000
3426922991611,3cyclictest3493060-21cpu22:15:112
3426917991614,1cyclictest3056-21PK-Backend23:31:081
3426917991614,1cyclictest3056-21PK-Backend23:00:191
3426917991614,1cyclictest3056-21PK-Backend21:02:511
3426917991614,1cyclictest3056-21PK-Backend19:34:501
3426917991614,1cyclictest3056-21PK-Backend19:34:501
342691799161,14cyclictest3439926-21grep19:45:121
3426915991614,1cyclictest0-21swapper/023:00:490
3426915991614,1cyclictest0-21swapper/021:12:140
3426915991614,1cyclictest0-21swapper/000:28:160
3426915991612,3cyclictest3505854-21tail22:50:180
3426915991611,4cyclictest3502847-21taskset22:40:500
342691799159,5cyclictest3502092-21diskstats22:40:111
3426917991513,1cyclictest3056-21PK-Backend00:08:191
3426915991513,1cyclictest0-21swapper/022:24:060
3426915991513,1cyclictest0-21swapper/022:24:060
3426915991511,3cyclictest918-21rs:main0
3426915991511,2cyclictest13-21ksoftirqd/020:50:180
3426917991410,3cyclictest3056-21PK-Backend19:58:201
3426917991410,3cyclictest3056-21PK-Backend19:20:271
3426917991311,1cyclictest3056-21PK-Backend20:17:091
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional