You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 17:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Tue Aug 25, 2026 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
235159321100,3sleep30-21swapper/308:25:153
232306527658,13sleep10-21swapper/107:07:231
232293327441,26sleep30-21swapper/307:05:403
232327426748,11sleep20-21swapper/207:10:012
232297726242,14sleep00-21swapper/007:06:140
23382952490,5sleep0161rcuc/007:50:000
23317692470,2sleep10-21swapper/107:30:131
24333712410,3sleep30-21swapper/312:15:203
24161592400,3sleep20-21swapper/211:30:002
139099380,4rtkit-daemon1389-21rtkit-daemon09:10:241
23267482350,2sleep20-21swapper/207:15:222
2323423993412,5cyclictest2369740-21ls09:15:222
139099330,3rtkit-daemon42-21ksoftirqd/310:24:413
2323423993111,5cyclictest2379832-21df09:45:142
232342399299,5cyclictest978-21snmpd07:55:122
232342399286,5cyclictest2364825-21/usr/sbin/munin09:05:002
2323423992810,6cyclictest2342650-21cat08:00:132
23250692270,3chrt3056-21PK-Backend07:15:001
232342399271,9cyclictest2385788-21python310:00:212
232342399271,8cyclictest2397048-21cron10:35:002
232342399262,8cyclictest2395065-21munin-run10:30:002
232342399261,11cyclictest2421326-21latency_hist11:45:002
232342399257,3cyclictest0-21swapper/211:20:132
232342399257,3cyclictest0-21swapper/209:30:182
2323423992512,3cyclictest978-21snmpd10:55:472
232342399251,13cyclictest2371385-21python309:20:202
232342399249,3cyclictest2402250-21cron10:50:002
2323423992418,5cyclictest0-21swapper/208:56:142
2323423992417,1cyclictest0-21swapper/207:23:212
2323423992414,7cyclictest2338263-21cron07:50:002
139099240,3rtkit-daemon1389-21rtkit-daemon10:17:012
23718592230,3sleep00-21swapper/009:23:350
232342999230,21cyclictest0-21swapper/310:50:013
232342399239,10cyclictest436-20systemd-journal08:15:002
232342399232,7cyclictest2375734-21apt09:35:112
232342399232,7cyclictest2335942-21python307:40:222
2323416992315,6cyclictest436-20systemd-journal08:55:011
232341699230,21cyclictest1389-21rtkit-daemon07:20:111
232341499231,2cyclictest2156-21gnome-shell12:38:090
232342999225,15cyclictest2419601-21/usr/sbin/munin11:40:003
2323429992220,1cyclictest0-21swapper/312:35:463
2323429992220,1cyclictest0-21swapper/310:40:213
2323429992220,1cyclictest0-21swapper/309:29:573
2323429992220,1cyclictest0-21swapper/307:42:103
2323429992219,2cyclictest3056-21PK-Backend07:27:363
232342999220,5cyclictest42-21ksoftirqd/310:58:093
232342999220,5cyclictest42-21ksoftirqd/309:42:053
2323423992220,1cyclictest0-21swapper/212:05:412
2323423992220,1cyclictest0-21swapper/208:54:392
2323423992220,1cyclictest0-21swapper/208:31:472
2323423992217,4cyclictest0-21swapper/212:12:242
2323416992219,2cyclictest3056-21PK-Backend07:48:541
2323416992213,7cyclictest2424789-21apt-get11:55:001
2323414992220,1cyclictest0-21swapper/010:45:140
2323414992220,1cyclictest0-21swapper/010:22:560
2323414992220,1cyclictest0-21swapper/008:57:190
2323414992220,1cyclictest0-21swapper/008:41:440
2323414992220,1cyclictest0-21swapper/007:23:230
2323414992220,1cyclictest0-21swapper/007:18:430
2323414992217,3cyclictest2409200-21munin-run11:10:000
232341499221,3cyclictest2331032-21apt-get07:29:590
232341499220,21cyclictest3056-21PK-Backend07:33:100
232341499220,21cyclictest0-21swapper/010:36:350
232341499220,21cyclictest0-21swapper/010:27:100
139099220,15rtkit-daemon0-21swapper/209:12:342
2323429992119,1cyclictest3056-21PK-Backend11:55:513
2323429992119,1cyclictest3056-21PK-Backend11:30:573
2323429992119,1cyclictest3056-21PK-Backend07:23:303
2323429992119,1cyclictest0-21swapper/312:25:393
2323429992119,1cyclictest0-21swapper/312:20:113
2323429992119,1cyclictest0-21swapper/310:25:423
2323429992119,1cyclictest0-21swapper/310:04:263
2323429992119,1cyclictest0-21swapper/309:56:483
2323429992119,1cyclictest0-21swapper/308:52:463
2323429992119,1cyclictest0-21swapper/307:11:013
232342999211,6cyclictest42-21ksoftirqd/310:08:323
2323429992113,6cyclictest883-21dbus-daemon10:20:003
2323429992112,7cyclictest2412654-21cron11:19:593
232342999210,20cyclictest3056-21PK-Backend10:37:443
232342999210,20cyclictest0-21swapper/310:32:303
232342999210,19cyclictest2420565-21ntp_kernel_err11:40:183
232342999210,12cyclictest0-21swapper/309:48:423
2323423992119,1cyclictest0-21swapper/211:57:102
2323423992119,1cyclictest0-21swapper/211:16:272
2323423992119,1cyclictest0-21swapper/208:36:072
2323423992119,1cyclictest0-21swapper/208:36:072
2323423992114,1cyclictest0-21swapper/212:35:432
2323423992114,1cyclictest0-21swapper/208:17:552
2323423992113,5cyclictest2384624-21latency_hist10:00:012
232342399210,20cyclictest0-21swapper/211:52:302
232342399210,20cyclictest0-21swapper/207:36:522
232341699210,20cyclictest3056-21PK-Backend12:35:001
232341699210,20cyclictest3056-21PK-Backend11:41:001
232341699210,19cyclictest3056-21PK-Backend12:05:401
232341699210,19cyclictest3056-21PK-Backend09:53:051
2323414992119,1cyclictest3056-21PK-Backend09:15:230
2323414992119,1cyclictest0-21swapper/011:35:500
2323414992119,1cyclictest0-21swapper/011:34:480
2323414992119,1cyclictest0-21swapper/011:25:430
2323414992119,1cyclictest0-21swapper/010:18:440
2323414992119,1cyclictest0-21swapper/010:14:310
2323414992119,1cyclictest0-21swapper/009:38:420
2323414992119,1cyclictest0-21swapper/009:03:280
2323414992119,1cyclictest0-21swapper/008:13:260
2323414992118,2cyclictest0-21swapper/011:55:030
232341499210,20cyclictest0-21swapper/012:03:210
232341499210,20cyclictest0-21swapper/010:57:300
232341499210,20cyclictest0-21swapper/010:50:280
232341499210,20cyclictest0-21swapper/008:30:220
232341499210,20cyclictest0-21swapper/008:16:430
232341499210,19cyclictest13-21ksoftirqd/009:05:140
139099210,14rtkit-daemon3056-21PK-Backend07:32:452
232342999205,13cyclictest2347068-21users08:10:233
232342999204,14cyclictest2438126-21hddtemp_smartct12:30:173
232342999203,16cyclictest884-21gmain09:05:493
2323429992018,1cyclictest0-21swapper/312:01:193
2323429992018,1cyclictest0-21swapper/311:50:243
2323429992018,1cyclictest0-21swapper/311:05:083
2323429992018,1cyclictest0-21swapper/309:53:233
2323429992018,1cyclictest0-21swapper/308:31:303
2323429992018,1cyclictest0-21swapper/308:17:053
2323429992018,1cyclictest0-21swapper/307:35:273
2323429992017,2cyclictest3056-21PK-Backend11:12:593
232342999200,6cyclictest42-21ksoftirqd/308:05:183
232342999200,6cyclictest42-21ksoftirqd/307:30:333
232342999200,5cyclictest42-21ksoftirqd/308:59:473
232342999200,5cyclictest42-21ksoftirqd/307:47:023
232342999200,1cyclictest0-21swapper/312:05:583
232342999200,1cyclictest0-21swapper/311:47:333
232342999200,19cyclictest0-21swapper/311:20:023
232342999200,19cyclictest0-21swapper/311:01:303
232342999200,19cyclictest0-21swapper/309:20:113
232342999200,19cyclictest0-21swapper/309:01:533
232342999200,19cyclictest0-21swapper/308:40:103
232342999200,19cyclictest0-21swapper/307:16:553
2323423992018,1cyclictest0-21swapper/212:31:032
2323423992018,1cyclictest0-21swapper/211:00:182
2323423992018,1cyclictest0-21swapper/210:38:432
2323423992018,1cyclictest0-21swapper/210:12:472
2323423992018,1cyclictest0-21swapper/209:05:342
2323423992018,1cyclictest0-21swapper/208:29:392
2323423992012,6cyclictest2423279-21cut11:50:012
2323423992011,7cyclictest2350201-21ntp_states08:20:182
232342399200,1cyclictest0-21swapper/209:26:292
232342399200,1cyclictest0-21swapper/208:08:012
232342399200,19cyclictest0-21swapper/212:15:142
232342399200,19cyclictest0-21swapper/212:01:502
232342399200,19cyclictest0-21swapper/210:43:082
232342399200,19cyclictest0-21swapper/207:52:212
2323416992018,1cyclictest0-21swapper/112:39:271
2323416992018,1cyclictest0-21swapper/110:38:201
2323416992017,2cyclictest3056-21PK-Backend09:44:531
2323416992012,6cyclictest2386939-21cat10:05:131
232341699200,19cyclictest3056-21PK-Backend12:10:201
232341699200,19cyclictest3056-21PK-Backend12:01:001
232341699200,19cyclictest3056-21PK-Backend11:17:351
232341699200,19cyclictest3056-21PK-Backend10:18:541
232341699200,19cyclictest3056-21PK-Backend10:14:411
232341699200,19cyclictest3056-21PK-Backend08:46:061
232341699200,19cyclictest3056-21PK-Backend08:02:331
232341699200,19cyclictest0-21swapper/112:25:541
232341699200,19cyclictest0-21swapper/110:58:141
2323414992018,1cyclictest0-21swapper/012:33:430
2323414992018,1cyclictest0-21swapper/012:28:150
2323414992018,1cyclictest0-21swapper/011:16:030
2323414992018,1cyclictest0-21swapper/010:42:370
2323414992018,1cyclictest0-21swapper/010:33:580
2323414992018,1cyclictest0-21swapper/008:52:460
2323414992018,1cyclictest0-21swapper/008:37:260
2323414992018,1cyclictest0-21swapper/008:37:260
2323414992018,1cyclictest0-21swapper/007:59:460
2323414992016,3cyclictest2351358-21grep08:25:130
2323414992014,3cyclictest2424816-21idleruntime-cro11:55:010
232341499200,1cyclictest3056-21PK-Backend09:25:440
232341499200,19cyclictest3056-21PK-Backend11:45:570
232341499200,19cyclictest0-21swapper/011:11:360
232341499200,19cyclictest0-21swapper/008:22:440
232341499200,19cyclictest0-21swapper/007:36:080
2323429991917,1cyclictest0-21swapper/308:35:513
2323429991917,1cyclictest0-21swapper/308:35:503
2323429991914,3cyclictest2340538-21apt07:55:113
2323429991912,5cyclictest436-20systemd-journal10:14:493
232342999190,5cyclictest42-21ksoftirqd/308:04:413
232342999190,1cyclictest0-21swapper/312:13:293
232342999190,18cyclictest0-21swapper/309:35:463
232342399192,12cyclictest2358131-21apt08:45:122
2323423991917,1cyclictest0-21swapper/211:38:312
2323423991917,1cyclictest0-21swapper/209:50:422
232342399190,1cyclictest0-21swapper/212:28:132
232342399190,1cyclictest0-21swapper/208:40:262
232342399190,18cyclictest0-21swapper/207:10:362
2323416991917,1cyclictest3056-21PK-Backend11:26:261
2323416991917,1cyclictest3056-21PK-Backend10:31:311
2323416991917,1cyclictest3056-21PK-Backend08:18:211
2323416991917,1cyclictest3056-21PK-Backend07:51:171
2323416991917,1cyclictest0-21swapper/112:22:161
2323416991917,1cyclictest0-21swapper/111:12:211
2323416991917,1cyclictest0-21swapper/107:28:131
2323416991912,5cyclictest2422622-21sed11:45:211
232341699190,1cyclictest3056-21PK-Backend11:35:321
232341699190,1cyclictest3056-21PK-Backend10:51:111
232341699190,1cyclictest3056-21PK-Backend09:35:541
232341699190,1cyclictest3056-21PK-Backend09:01:061
232341699190,1cyclictest3056-21PK-Backend08:36:051
232341699190,1cyclictest3056-21PK-Backend08:36:051
232341699190,1cyclictest3056-21PK-Backend07:55:291
232341699190,1cyclictest0-21swapper/111:31:541
232341699190,1cyclictest0-21swapper/111:00:031
232341699190,1cyclictest0-21swapper/108:43:081
232341699190,18cyclictest3056-21PK-Backend10:48:351
232341699190,18cyclictest3056-21PK-Backend10:28:061
232341699190,18cyclictest3056-21PK-Backend08:06:121
2323414991917,1cyclictest0-21swapper/009:11:250
2323414991917,1cyclictest0-21swapper/007:13:440
2323414991916,2cyclictest3056-21PK-Backend11:20:300
2323414991913,5cyclictest2374395-21fw_packets09:30:170
2323414991913,4cyclictest2336541-21cron07:45:000
2323414991911,6cyclictest2345459-21apt-get08:10:000
232341499190,1cyclictest0-21swapper/012:06:120
232341499190,1cyclictest0-21swapper/010:01:070
232341499190,1cyclictest0-21swapper/009:50:180
232341499190,18cyclictest0-21swapper/009:55:180
2323429991812,5cyclictest2339250-21ntpq07:50:183
232342999180,5cyclictest3056-21PK-Backend09:19:083
232342999180,5cyclictest2374004-21apt09:30:123
2323423991816,1cyclictest0-21swapper/211:14:382
2323423991816,1cyclictest0-21swapper/210:51:122
2323423991816,1cyclictest0-21swapper/210:05:092
2323423991812,5cyclictest2392229-21diskstats10:20:122
2323423991810,6cyclictest2377834-21df_inode09:40:152
2323416991816,1cyclictest3056-21PK-Backend12:16:491
2323416991816,1cyclictest3056-21PK-Backend09:58:521
2323416991816,1cyclictest3056-21PK-Backend08:11:111
2323416991816,1cyclictest3056-21PK-Backend07:15:551
2323416991816,1cyclictest0-21swapper/111:07:061
2323416991816,1cyclictest0-21swapper/108:25:441
2323416991815,2cyclictest3056-21PK-Backend07:39:011
2323416991811,5cyclictest2371959-21sshd09:24:501
2323414991816,1cyclictest0-21swapper/012:16:330
2323414991816,1cyclictest0-21swapper/012:10:030
2323414991816,1cyclictest0-21swapper/011:40:300
2323414991816,1cyclictest0-21swapper/011:01:570
2323414991816,1cyclictest0-21swapper/008:03:240
2323414991815,2cyclictest3056-21PK-Backend08:49:000
2323414991811,6cyclictest2387396-21ntp_states10:05:200
2323414991811,5cyclictest2378140-21cut09:40:190
232342999171,5cyclictest42-21ksoftirqd/311:27:243
2323429991710,5cyclictest2359760-21sed08:50:003
232342999170,7cyclictest42-21ksoftirqd/309:10:173
232342399179,6cyclictest2329703-21grep07:25:132
2323423991715,1cyclictest0-21swapper/211:33:512
2323423991715,1cyclictest0-21swapper/211:08:242
232341699178,7cyclictest2375463-21cron09:34:591
2323416991715,1cyclictest3056-21PK-Backend08:24:231
2323416991715,1cyclictest0-21swapper/111:57:221
2323416991715,1cyclictest0-21swapper/111:22:001
2323416991714,2cyclictest3056-21PK-Backend09:08:031
232342999160,7cyclictest42-21ksoftirqd/310:50:293
232342999160,3cyclictest2350044-21memory08:20:163
2323423991614,1cyclictest0-21swapper/212:24:342
2323416991614,1cyclictest3056-21PK-Backend07:42:451
2323416991614,1cyclictest0-21swapper/110:40:101
232341699160,1cyclictest3056-21PK-Backend10:23:541
232341699159,4cyclictest2352912-21acpi08:30:141
2323416991512,2cyclictest3056-21PK-Backend10:03:531
232341699150,13cyclictest2363281-21date09:00:021
232341499158,6cyclictest2434714-21cat12:20:170
2323414991513,1cyclictest0-21swapper/009:49:300
2323414991510,3cyclictest1095-21runrttasks07:50:200
2323416991412,1cyclictest0-21swapper/109:25:331
2323416991410,3cyclictest2368876-21cpuspeed_turbos09:15:121
232341699117,3cyclictest3056-21PK-Backend09:45:091
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional