You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 19:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Thu Apr 02, 2026 12:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3025121200,3sleep0161rcuc/009:35:160
8279121090,4sleep30-21swapper/312:05:013
9421021040,3sleep00-21swapper/012:35:160
139099800,70rtkit-daemon0-21swapper/007:07:360
417698027354,14sleep10-21swapper/107:09:471
417627126644,14sleep20-21swapper/207:05:182
417542826539,20sleep30-21swapper/307:05:033
582652270,5sleep13056-21PK-Backend10:55:011
417714599273,8cyclictest21952-21taskset09:12:363
866412260,4sleep13056-21PK-Backend12:15:011
4177141992622,3cyclictest34-21ksoftirqd/210:05:352
724752250,4sleep30-21swapper/311:35:013
417714599257,5cyclictest0-21swapper/309:05:033
417714599255,11cyclictest16241-21seq08:57:443
417714599254,8cyclictest2156-21gnome-shell11:26:093
4177145992520,4cyclictest0-21swapper/310:13:253
417714599248,14cyclictest92475-21memory12:30:163
417714599247,15cyclictest74440-21cpuspeed_turbos11:40:123
125762240,3chrt3056-21PK-Backend08:45:211
9092230,4sleep00-21swapper/008:15:130
4177145992321,1cyclictest0-21swapper/308:37:003
4177145992317,5cyclictest4177508-21mktemp07:11:563
417714599230,21cyclictest17020-21cpuspeed09:00:123
417714199231,21cyclictest39894-21kworker/2:210:05:012
417714199230,21cyclictest12241-21ntp_states08:45:182
417713899230,21cyclictest3056-21PK-Backend11:28:381
139099230,4rtkit-daemon1389-21rtkit-daemon12:09:090
139099230,4rtkit-daemon1389-21rtkit-daemon12:09:090
139099230,4rtkit-daemon1389-21rtkit-daemon11:41:591
417714599227,13cyclictest33828-21ntp_states09:45:183
417714599226,14cyclictest4192812-21grep08:10:123
417714599226,14cyclictest4178208-21ls07:15:033
417714599226,14cyclictest37411-21python309:55:193
417714599223,6cyclictest61880-21apt11:05:123
4177145992220,1cyclictest0-21swapper/310:53:283
4177145992220,1cyclictest0-21swapper/310:38:543
4177145992220,1cyclictest0-21swapper/309:34:193
4177145992220,1cyclictest0-21swapper/308:06:033
417714599220,20cyclictest4190116-21sshd08:00:163
4177141992220,1cyclictest0-21swapper/210:13:262
4177141992219,2cyclictest0-21swapper/209:30:202
4177141992215,5cyclictest886-21NetworkManager09:59:472
417714199220,3cyclictest33477-21fw_packets09:45:132
4177138992219,2cyclictest3056-21PK-Backend10:06:121
4177138992219,2cyclictest3056-21PK-Backend07:54:101
417713899220,21cyclictest3056-21PK-Backend07:17:271
417713899220,20cyclictest3056-21PK-Backend12:33:551
417713899220,20cyclictest3056-21PK-Backend11:35:351
417713899220,20cyclictest3056-21PK-Backend11:30:411
417713899220,20cyclictest3056-21PK-Backend09:46:181
417714599218,11cyclictest25021-21/usr/sbin/munin09:20:203
417714599216,13cyclictest1304-21ntp_states08:15:173
417714599212,6cyclictest44024-21apt10:15:123
4177145992119,1cyclictest411rcuc/307:23:263
4177145992119,1cyclictest0-21swapper/312:19:533
4177145992119,1cyclictest0-21swapper/311:51:213
4177145992119,1cyclictest0-21swapper/309:52:513
4177145992119,1cyclictest0-21swapper/308:53:503
4177145992119,1cyclictest0-21swapper/307:42:053
4177145992110,9cyclictest7541-21unin-run08:35:003
417714599210,8cyclictest65339-21/usr/sbin/munin11:15:103
417714599210,20cyclictest0-21swapper/311:48:153
417714599210,20cyclictest0-21swapper/307:35:143
417714599210,20cyclictest0-21swapper/307:34:003
4177141992114,6cyclictest58154-21cron10:55:002
4177141992114,5cyclictest46229-21cat10:20:112
4177141992114,5cyclictest18430-21perf09:05:012
4177141992114,5cyclictest15144-21expr08:55:122
4177138992119,1cyclictest3056-21PK-Backend12:01:031
4177138992119,1cyclictest3056-21PK-Backend11:54:071
4177138992119,1cyclictest3056-21PK-Backend10:58:381
4177138992119,1cyclictest3056-21PK-Backend10:16:401
4177138992119,1cyclictest3056-21PK-Backend09:31:301
4177138992119,1cyclictest3056-21PK-Backend08:39:401
4177138992118,2cyclictest3056-21PK-Backend09:55:441
417713899210,20cyclictest3056-21PK-Backend11:22:441
417713899210,20cyclictest3056-21PK-Backend09:15:431
417713899210,19cyclictest3056-21PK-Backend11:05:481
417713899210,19cyclictest3056-21PK-Backend08:41:431
417713899210,19cyclictest3056-21PK-Backend08:23:041
417713899210,19cyclictest3056-21PK-Backend08:15:271
417713899210,19cyclictest3056-21PK-Backend07:41:001
4177135992119,1cyclictest0-21swapper/011:52:470
4177135992119,1cyclictest0-21swapper/010:57:380
417713599210,20cyclictest0-21swapper/012:16:060
139099210,4rtkit-daemon1389-21rtkit-daemon10:45:351
417714599209,9cyclictest51012-21rm10:35:013
417714599206,13cyclictest73662-21chrt11:35:493
417714599206,12cyclictest90359-21cstates12:25:123
4177145992018,1cyclictest0-21swapper/312:23:453
4177145992018,1cyclictest0-21swapper/311:21:193
4177145992018,1cyclictest0-21swapper/310:05:483
4177145992018,1cyclictest0-21swapper/309:40:543
4177145992014,4cyclictest30693-21users09:35:223
417714599200,1cyclictest0-21swapper/307:56:453
417714599200,1cyclictest0-21swapper/307:26:573
417714599200,19cyclictest0-21swapper/312:36:563
417714599200,19cyclictest0-21swapper/312:11:203
417714599200,19cyclictest0-21swapper/312:06:283
417714599200,19cyclictest0-21swapper/312:06:273
417714599200,19cyclictest0-21swapper/311:59:183
417714599200,19cyclictest0-21swapper/310:57:333
417714599200,19cyclictest0-21swapper/308:46:263
417714599200,19cyclictest0-21swapper/308:23:013
4177141992014,5cyclictest68530-21cron11:24:592
4177141992014,4cyclictest978-21snmpd11:55:152
4177141992012,6cyclictest1-21systemd11:50:012
4177141992010,8cyclictest79203-21cron11:55:002
4177138992018,1cyclictest3056-21PK-Backend12:26:381
4177138992018,1cyclictest3056-21PK-Backend12:23:471
4177138992018,1cyclictest3056-21PK-Backend11:45:221
4177138992018,1cyclictest3056-21PK-Backend11:03:441
4177138992018,1cyclictest3056-21PK-Backend08:05:331
4177138992018,1cyclictest3056-21PK-Backend08:03:171
4177138992018,1cyclictest3056-21PK-Backend07:46:341
4177138992018,1cyclictest3056-21PK-Backend07:33:091
4177138992018,1cyclictest3056-21PK-Backend07:22:471
417713899200,19cyclictest3056-21PK-Backend11:17:511
417713899200,19cyclictest3056-21PK-Backend08:26:211
417713899200,19cyclictest3056-21PK-Backend08:12:091
417713899200,19cyclictest3056-21PK-Backend07:58:431
417713899200,19cyclictest0-21swapper/107:35:251
4177135992018,1cyclictest0-21swapper/011:24:080
417713599200,1cyclictest0-21swapper/012:27:480
417713599200,19cyclictest0-21swapper/012:24:100
417713599200,19cyclictest0-21swapper/011:31:500
417713599200,19cyclictest0-21swapper/010:52:440
417713599200,19cyclictest0-21swapper/010:27:160
417713599200,19cyclictest0-21swapper/010:19:180
417713599200,19cyclictest0-21swapper/009:54:110
417713599200,19cyclictest0-21swapper/009:22:540
417713599200,19cyclictest0-21swapper/008:48:270
417713599200,19cyclictest0-21swapper/007:56:150
417713599200,19cyclictest0-21swapper/007:53:590
417713599200,19cyclictest0-21swapper/007:47:370
417713599200,19cyclictest0-21swapper/007:13:160
417713599200,18cyclictest3056-21PK-Backend08:55:230
417714599196,11cyclictest4280-21/usr/sbin/munin08:25:213
4177145991917,1cyclictest0-21swapper/307:52:123
4177145991914,3cyclictest978-21snmpd07:47:443
4177145991912,5cyclictest24050-21ls09:20:013
417714599190,1cyclictest0-21swapper/311:10:033
417714599190,16cyclictest53951-21ntp_states10:40:163
417714199199,7cyclictest22789-21df_inode09:15:122
4177141991917,1cyclictest0-21swapper/211:09:022
4177141991916,2cyclictest1095-21runrttasks09:29:512
4177141991914,4cyclictest49209-21cron10:30:002
4177141991914,4cyclictest34-21ksoftirqd/208:25:202
4177141991914,3cyclictest978-21snmpd09:14:442
4177141991913,5cyclictest7928-21mktemp08:35:012
4177141991913,4cyclictest92057-21cpuspeed_turbos12:30:122
4177141991912,5cyclictest94600-21sed12:35:212
4177141991912,5cyclictest65080-21/usr/sbin/munin11:15:002
4177141991912,5cyclictest54100-21expr10:40:202
4177141991910,7cyclictest4194227-21perf08:15:002
4177138991917,1cyclictest3056-21PK-Backend10:41:281
4177138991917,1cyclictest3056-21PK-Backend07:26:191
4177138991917,1cyclictest3056-21PK-Backend07:12:541
4177138991916,2cyclictest3056-21PK-Backend10:26:051
4177138991916,2cyclictest3056-21PK-Backend09:10:361
417713599199,4cyclictest13-21ksoftirqd/010:40:520
4177135991917,1cyclictest0-21swapper/011:37:320
4177135991917,1cyclictest0-21swapper/010:09:170
4177135991917,1cyclictest0-21swapper/009:34:570
4177135991917,1cyclictest0-21swapper/009:10:030
4177135991917,1cyclictest0-21swapper/008:05:540
4177135991912,6cyclictest1-21systemd07:32:340
417713599190,1cyclictest3056-21PK-Backend11:25:090
417713599190,1cyclictest3056-21PK-Backend09:00:160
417713599190,1cyclictest0-21swapper/011:05:340
417713599190,1cyclictest0-21swapper/011:02:310
417713599190,1cyclictest0-21swapper/010:38:030
417713599190,1cyclictest0-21swapper/009:56:130
417713599190,1cyclictest0-21swapper/009:46:130
417713599190,1cyclictest0-21swapper/009:19:500
417713599190,1cyclictest0-21swapper/008:00:330
417713599190,18cyclictest3056-21PK-Backend11:40:220
417713599190,18cyclictest0-21swapper/010:00:180
417713599190,18cyclictest0-21swapper/009:42:060
417713599190,18cyclictest0-21swapper/008:21:430
417713599190,18cyclictest0-21swapper/007:17:010
417714599189,7cyclictest60540-21smartctl11:00:163
417714599189,7cyclictest55283-21expr10:45:113
417714599187,9cyclictest39759-21idleruntime-cro10:05:013
4177145991816,1cyclictest0-21swapper/310:24:073
4177145991812,5cyclictest26043-21expr09:25:133
4177145991811,5cyclictest48614-21python310:25:203
417714199189,7cyclictest34-21ksoftirqd/207:35:072
417714199187,10cyclictest25613-21sshd09:24:592
417714199184,13cyclictest4185751-21cron07:44:592
417714199182,15cyclictest1095-21runrttasks12:19:002
417714199182,15cyclictest1095-21runrttasks07:23:522
417714199182,14cyclictest61597-21cron11:05:002
4177141991816,1cyclictest0-21swapper/211:27:282
4177141991816,1cyclictest0-21swapper/207:50:362
4177141991814,3cyclictest978-21snmpd09:52:442
4177141991813,4cyclictest56382-21seq10:49:502
4177141991813,4cyclictest2156-21gnome-shell08:04:092
4177141991811,5cyclictest88815-21which12:20:132
4177141991811,5cyclictest84465-21dpkg12:10:002
4177141991811,5cyclictest84465-21dpkg12:10:002
4177141991811,5cyclictest66962-21wc11:20:002
4177141991811,5cyclictest58483-21cat10:55:112
4177141991811,5cyclictest13682-21cat08:50:152
4177141991810,6cyclictest6787-21ntp_states08:30:192
4177141991810,6cyclictest10172-21which08:40:152
4177138991816,1cyclictest3056-21PK-Backend11:10:551
4177138991814,2cyclictest3056-21PK-Backend09:05:291
4177138991813,3cyclictest1095-21runrttasks12:05:211
4177138991813,3cyclictest1095-21runrttasks12:05:201
4177135991816,1cyclictest3056-21PK-Backend08:50:300
4177135991816,1cyclictest0-21swapper/012:14:160
4177135991816,1cyclictest0-21swapper/009:27:000
4177135991816,1cyclictest0-21swapper/007:25:120
4177135991813,3cyclictest7513-21latency_hist08:35:010
4177135991813,3cyclictest4192910-21cpuspeed_turbos08:10:130
4177135991811,5cyclictest51218-21cut10:35:010
4177145991713,3cyclictest10250-21irqstats08:40:153
417714199179,6cyclictest51873-21fw_packets10:35:142
417714199179,6cyclictest34-21ksoftirqd/208:05:072
4177141991715,1cyclictest0-21swapper/212:00:262
4177141991715,1cyclictest0-21swapper/207:30:222
4177141991715,1cyclictest0-21swapper/207:29:202
4177141991715,1cyclictest0-21swapper/207:16:502
4177141991715,1cyclictest0-21swapper/207:12:092
4177141991712,3cyclictest85645-21netstat12:10:162
4177141991712,3cyclictest4186727-21netstat07:45:132
4177141991712,3cyclictest19728-21open_inodes09:05:192
4177141991710,6cyclictest90866-21ntpq12:25:172
4177141991710,6cyclictest1201-21cat08:15:152
4177141991710,5cyclictest918-21rs:main2
4177141991710,5cyclictest71513-21ntpq11:30:182
4177141991710,5cyclictest29092-21apt-get09:35:012
4177138991715,1cyclictest3056-21PK-Backend10:35:191
4177138991715,1cyclictest3056-21PK-Backend09:38:551
4177138991715,1cyclictest0-21swapper/109:00:221
417713899171,11cyclictest36288-21latency_hist09:55:001
4177135991715,1cyclictest0-21swapper/009:06:100
4177135991715,1cyclictest0-21swapper/008:41:440
4177135991710,5cyclictest66062-21ntp_kernel_pll_11:15:170
417713599170,16cyclictest0-21swapper/010:11:210
417713599170,16cyclictest0-21swapper/008:26:510
417714199169,5cyclictest74997-21ntpq11:40:192
417714199169,5cyclictest72889-21expr11:35:132
417714199168,6cyclictest32104-21awk09:40:182
4177138991614,1cyclictest3056-21PK-Backend12:19:091
4177138991614,1cyclictest3056-21PK-Backend11:56:091
4177138991614,1cyclictest3056-21PK-Backend10:30:111
4177138991614,1cyclictest3056-21PK-Backend09:21:041
4177138991613,2cyclictest3056-21PK-Backend12:38:351
4177138991610,4cyclictest16519-21sed09:00:001
4177138991610,4cyclictest13646-21basename08:50:151
417713599166,4cyclictest13-21ksoftirqd/010:48:470
4177135991612,3cyclictest91871-21/usr/sbin/munin12:30:150
4177135991610,5cyclictest9000-21sed08:35:200
417714199158,5cyclictest2616-21cat08:20:112
417714199157,6cyclictest44023-21sh10:15:112
4177141991511,3cyclictest50392-21switchtime10:30:192
417713899152,12cyclictest39756-21cron10:05:011
417713599159,5cyclictest957-21gdbus07:20:300
417713599159,5cyclictest4185760-21cron07:45:000
417713599159,4cyclictest80782-21systemd-udevd11:58:110
4177135991513,1cyclictest0-21swapper/011:48:550
4177135991513,1cyclictest0-21swapper/011:13:320
4177135991513,1cyclictest0-21swapper/010:22:230
4177135991511,3cyclictest4184858-21dump-pmu-power07:40:000
417713899141,9cyclictest7507-21perf08:35:001
4177138991412,1cyclictest0-21swapper/109:43:141
4177138991410,3cyclictest3056-21PK-Backend10:11:431
417713899140,6cyclictest3056-21PK-Backend10:20:191
417713599137,5cyclictest81327-21ls12:00:120
417713899128,3cyclictest3056-21PK-Backend09:26:231
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional