You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-24 - 19:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu Sep 24, 2026 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
259074821310,3sleep13056-21PK-Backend11:35:131
250687221220,4sleep10-21swapper/107:35:131
251249121020,2sleep20-21swapper/207:50:182
25731612970,4sleep03056-21PK-Backend10:45:030
249504527446,18sleep30-21swapper/307:05:103
139099730,59rtkit-daemon0-21swapper/107:08:371
25696732690,4sleep33056-21PK-Backend10:35:123
249624226545,14sleep00-21swapper/007:06:490
249616126343,13sleep20-21swapper/207:05:472
25804942520,4sleep20-21swapper/211:05:142
24985722400,2sleep20-21swapper/207:15:012
25764902390,3sleep33056-21PK-Backend10:55:013
25995522300,11sleep342-21ksoftirqd/312:00:123
25405642270,3sleep10-21swapper/109:10:201
25960512260,11sleep234-21ksoftirqd/211:50:142
25860522260,3sleep13056-21PK-Backend11:20:161
25499132260,3sleep10-21swapper/109:40:001
25499132260,3sleep10-21swapper/109:39:591
25346342250,4sleep1251rcuc/108:55:131
249665699251,8cyclictest25136892expr07:55:122
25611542240,4chrt3056-21PK-Backend10:10:173
139099240,16rtkit-daemon13-21ksoftirqd/010:04:170
25833552230,7sleep2883-21dbus-daemon11:15:012
2496650992315,6cyclictest2595460-21latency_hist11:50:000
249665099231,20cyclictest2555090-21awk09:55:000
139099230,14rtkit-daemon0-21swapper/012:11:130
139099230,13rtkit-daemon0-21swapper/008:30:250
25461692220,4sleep32546170-21sleep09:30:003
249665899222,6cyclictest2499279-21apt-get07:15:043
2496658992220,1cyclictest3056-21PK-Backend09:46:423
2496658992219,2cyclictest3056-21PK-Backend10:58:133
2496658992214,6cyclictest2543685-21ntpq09:20:183
2496658992214,6cyclictest2498334-21munin-run07:15:003
249665899220,20cyclictest3056-21PK-Backend08:50:313
2496652992220,1cyclictest0-21swapper/111:52:111
2496652992220,1cyclictest0-21swapper/110:52:031
2496652992220,1cyclictest0-21swapper/107:22:551
2496652992217,3cyclictest2555346-21apt09:55:101
2496652992211,9cyclictest2547921-21unin-run09:35:001
2496650992214,6cyclictest436-20systemd-journal10:39:590
139099220,14rtkit-daemon0-21swapper/108:15:471
139099220,14rtkit-daemon0-21swapper/107:30:221
25364232210,5sleep00-21swapper/009:00:170
2496658992119,1cyclictest3056-21PK-Backend12:25:463
2496658992119,1cyclictest3056-21PK-Backend10:28:053
2496658992119,1cyclictest3056-21PK-Backend09:53:383
2496658992119,1cyclictest3056-21PK-Backend07:43:323
2496658992119,1cyclictest3056-21PK-Backend07:43:323
2496658992119,1cyclictest3056-21PK-Backend07:29:333
2496658992119,1cyclictest0-21swapper/312:07:413
2496658992119,1cyclictest0-21swapper/311:36:253
2496658992119,1cyclictest0-21swapper/311:05:293
249665899211,7cyclictest2586187-21python311:20:193
2496656992119,1cyclictest0-21swapper/208:02:242
2496656992114,5cyclictest2540941-21cron09:15:002
2496656992113,7cyclictest883-21dbus-daemon12:06:462
2496656992113,6cyclictest2584811-21cron11:17:002
2496656992113,6cyclictest2503545-21awk07:25:202
249665699210,20cyclictest0-21swapper/212:11:122
249665699210,20cyclictest0-21swapper/209:21:542
249665699210,20cyclictest0-21swapper/207:16:062
249665699210,1cyclictest0-21swapper/211:30:512
249665299212,16cyclictest2534030-21turbostat08:55:001
2496652992119,1cyclictest3056-21PK-Backend11:59:281
2496652992119,1cyclictest3056-21PK-Backend11:10:421
2496652992119,1cyclictest3056-21PK-Backend11:02:231
2496652992119,1cyclictest0-21swapper/112:19:431
2496652992119,1cyclictest0-21swapper/110:27:551
2496652992119,1cyclictest0-21swapper/109:47:341
2496652992119,1cyclictest0-21swapper/109:18:001
2496652992119,1cyclictest0-21swapper/108:48:531
2496652992119,1cyclictest0-21swapper/108:28:531
2496652992113,6cyclictest2609495-21sed12:30:011
2496652992113,6cyclictest2508660-21df07:40:141
2496652992113,6cyclictest2508660-21df07:40:141
249665299210,2cyclictest2528230-21python308:35:221
249665299210,2cyclictest2528230-21python308:35:221
249665299210,20cyclictest0-21swapper/112:13:141
249665299210,20cyclictest0-21swapper/110:36:141
249665299210,19cyclictest3056-21PK-Backend11:32:451
249665299210,19cyclictest3056-21PK-Backend07:12:141
2496650992119,1cyclictest0-21swapper/012:15:180
2496650992114,6cyclictest2563602-21sed10:15:230
249665099210,20cyclictest0-21swapper/011:24:150
249665099210,20cyclictest0-21swapper/011:09:060
249665099210,20cyclictest0-21swapper/008:22:320
249665099210,20cyclictest0-21swapper/008:01:570
249665099210,20cyclictest0-21swapper/007:29:320
139099210,15rtkit-daemon13-21ksoftirqd/010:07:410
139099210,15rtkit-daemon0-21swapper/012:07:280
139099210,14rtkit-daemon0-21swapper/109:23:261
139099210,14rtkit-daemon0-21swapper/107:15:101
139099210,14rtkit-daemon0-21swapper/007:35:420
139099210,13rtkit-daemon0-21swapper/011:40:420
2496658992018,1cyclictest3056-21PK-Backend08:37:203
2496658992018,1cyclictest3056-21PK-Backend08:37:203
2496658992018,1cyclictest0-21swapper/311:30:583
2496658992018,1cyclictest0-21swapper/311:27:203
2496658992018,1cyclictest0-21swapper/311:01:523
2496658992018,1cyclictest0-21swapper/307:24:063
249665899201,7cyclictest2583949-21df11:15:103
2496658992013,5cyclictest2596961-21sshd11:52:153
249665899200,19cyclictest3056-21PK-Backend12:23:103
249665899200,19cyclictest0-21swapper/311:46:193
249665899200,19cyclictest0-21swapper/311:10:573
249665899200,18cyclictest3056-21PK-Backend09:41:063
249665899200,18cyclictest3056-21PK-Backend07:45:353
2496656992018,1cyclictest0-21swapper/212:03:432
2496656992018,1cyclictest0-21swapper/210:51:022
2496656992018,1cyclictest0-21swapper/210:05:272
2496656992018,1cyclictest0-21swapper/209:52:422
2496656992018,1cyclictest0-21swapper/207:49:272
2496656992015,3cyclictest978-21snmpd09:44:132
2496656992014,4cyclictest2548094-21wc09:35:002
2496656992013,5cyclictest436-20systemd-journal11:04:142
249665699200,1cyclictest0-21swapper/209:00:572
249665699200,1cyclictest0-21swapper/208:58:212
249665699200,19cyclictest3056-21PK-Backend10:20:132
249665699200,19cyclictest0-21swapper/211:56:122
249665699200,19cyclictest0-21swapper/211:43:002
249665699200,19cyclictest0-21swapper/211:22:192
249665699200,19cyclictest0-21swapper/210:10:542
249665699200,19cyclictest0-21swapper/210:04:252
249665699200,19cyclictest0-21swapper/209:57:092
249665699200,19cyclictest0-21swapper/209:15:522
249665699200,19cyclictest0-21swapper/208:54:572
249665699200,19cyclictest0-21swapper/208:43:072
249665699200,19cyclictest0-21swapper/208:39:152
249665699200,19cyclictest0-21swapper/208:39:142
249665699200,19cyclictest0-21swapper/208:26:042
249665699200,19cyclictest0-21swapper/207:41:022
249665699200,19cyclictest0-21swapper/207:41:012
2496652992018,1cyclictest0-21swapper/112:38:081
2496652992018,1cyclictest0-21swapper/112:20:441
2496652992018,1cyclictest0-21swapper/110:40:541
2496652992018,1cyclictest0-21swapper/110:20:401
2496652992018,1cyclictest0-21swapper/109:51:471
2496652992018,1cyclictest0-21swapper/108:41:021
2496652992018,1cyclictest0-21swapper/108:23:251
2496652992018,1cyclictest0-21swapper/108:12:441
2496652992018,1cyclictest0-21swapper/108:05:411
2496652992015,3cyclictest2510395-21df07:45:131
249665299200,1cyclictest0-21swapper/110:10:321
249665299200,19cyclictest0-21swapper/111:49:221
249665299200,19cyclictest0-21swapper/110:17:491
249665299200,19cyclictest0-21swapper/110:01:261
249665299200,19cyclictest0-21swapper/109:40:581
249665299200,19cyclictest0-21swapper/108:02:021
249665299200,19cyclictest0-21swapper/107:27:481
2496650992018,1cyclictest0-21swapper/010:52:450
2496650992018,1cyclictest0-21swapper/008:35:290
2496650992018,1cyclictest0-21swapper/008:35:290
2496650992018,1cyclictest0-21swapper/007:16:550
2496650992017,2cyclictest0-21swapper/009:10:170
2496650992016,3cyclictest0-21swapper/008:25:210
2496650992012,6cyclictest883-21dbus-daemon12:35:010
2496650992012,6cyclictest883-21dbus-daemon12:35:000
2496650992011,7cyclictest2542614-21sshd09:19:130
249665099200,1cyclictest3056-21PK-Backend10:26:010
249665099200,1cyclictest0-21swapper/009:46:410
249665099200,1cyclictest0-21swapper/009:28:430
249665099200,1cyclictest0-21swapper/007:13:160
249665099200,19cyclictest0-21swapper/011:28:410
249665099200,19cyclictest0-21swapper/011:02:040
249665099200,19cyclictest0-21swapper/010:33:180
249665099200,19cyclictest0-21swapper/010:10:270
249665099200,19cyclictest0-21swapper/009:36:060
249665099200,19cyclictest0-21swapper/009:36:060
249665099200,19cyclictest0-21swapper/008:49:490
249665099200,19cyclictest0-21swapper/007:20:470
139099200,14rtkit-daemon3056-21PK-Backend10:59:411
139099200,14rtkit-daemon13-21ksoftirqd/010:22:090
139099200,14rtkit-daemon0-21swapper/109:05:361
139099200,14rtkit-daemon0-21swapper/007:53:200
139099200,14rtkit-daemon0-21swapper/007:31:480
139099200,13rtkit-daemon0-21swapper/011:35:500
139099200,13rtkit-daemon0-21swapper/008:53:070
2496658991917,1cyclictest3056-21PK-Backend10:30:423
2496658991917,1cyclictest3056-21PK-Backend10:24:413
2496658991917,1cyclictest3056-21PK-Backend10:03:383
2496658991917,1cyclictest3056-21PK-Backend07:55:423
2496658991917,1cyclictest3056-21PK-Backend07:30:353
2496658991916,2cyclictest3056-21PK-Backend09:58:383
2496658991916,2cyclictest3056-21PK-Backend08:55:513
2496658991916,2cyclictest3056-21PK-Backend08:17:053
249665899190,1cyclictest3056-21PK-Backend09:32:363
249665899190,1cyclictest3056-21PK-Backend08:28:413
249665899190,1cyclictest0-21swapper/311:55:593
249665899190,1cyclictest0-21swapper/311:41:533
249665899190,18cyclictest3056-21PK-Backend08:45:583
2496656991917,1cyclictest0-21swapper/212:38:092
2496656991917,1cyclictest0-21swapper/211:38:212
2496656991917,1cyclictest0-21swapper/210:15:332
2496656991917,1cyclictest0-21swapper/208:49:562
2496656991917,1cyclictest0-21swapper/208:33:342
2496656991917,1cyclictest0-21swapper/207:35:552
2496656991917,1cyclictest0-21swapper/207:22:502
2496656991914,3cyclictest436-20systemd-journal10:45:002
2496656991912,6cyclictest2587252-21expr11:25:122
249665699190,1cyclictest0-21swapper/207:30:482
249665699190,18cyclictest0-21swapper/210:56:442
249665699190,18cyclictest0-21swapper/210:45:212
249665699190,18cyclictest0-21swapper/209:35:052
249665699190,18cyclictest0-21swapper/209:35:052
249665699190,18cyclictest0-21swapper/208:19:352
249665699190,18cyclictest0-21swapper/208:13:182
2496652991917,1cyclictest3056-21PK-Backend12:05:581
2496652991917,1cyclictest0-21swapper/110:30:461
2496652991917,1cyclictest0-21swapper/110:08:431
2496652991917,1cyclictest0-21swapper/109:00:431
2496652991917,1cyclictest0-21swapper/107:59:251
249665299190,18cyclictest0-21swapper/112:30:521
249665299190,18cyclictest0-21swapper/112:30:521
2496650991917,1cyclictest0-21swapper/011:17:120
2496650991917,1cyclictest0-21swapper/011:13:330
2496650991917,1cyclictest0-21swapper/010:41:360
2496650991917,1cyclictest0-21swapper/008:57:260
2496650991917,1cyclictest0-21swapper/008:44:010
2496650991917,1cyclictest0-21swapper/007:57:300
2496650991917,1cyclictest0-21swapper/007:40:480
2496650991917,1cyclictest0-21swapper/007:40:480
2496650991912,5cyclictest2519397-21ntp_kernel_err08:10:180
2496650991911,6cyclictest2596092-21df11:50:140
249665099190,1cyclictest0-21swapper/012:22:350
139099190,13rtkit-daemon3056-21PK-Backend11:08:241
139099190,13rtkit-daemon0-21swapper/009:06:450
2496658991816,1cyclictest3056-21PK-Backend10:09:263
2496658991816,1cyclictest3056-21PK-Backend09:39:323
2496658991816,1cyclictest3056-21PK-Backend09:39:323
2496658991816,1cyclictest3056-21PK-Backend08:30:293
2496658991815,2cyclictest3056-21PK-Backend12:36:013
2496658991815,2cyclictest3056-21PK-Backend12:12:083
2496656991816,1cyclictest3056-21PK-Backend08:05:152
2496656991816,1cyclictest0-21swapper/212:29:522
2496656991816,1cyclictest0-21swapper/211:48:412
2496656991816,1cyclictest0-21swapper/209:49:172
2496656991816,1cyclictest0-21swapper/208:20:362
2496656991811,5cyclictest2545417-21awk09:25:182
249665699180,1cyclictest0-21swapper/212:15:522
249665299189,7cyclictest2512228-21cat07:50:141
2496652991816,1cyclictest0-21swapper/110:49:121
2496652991815,2cyclictest3056-21PK-Backend12:00:301
249665099189,7cyclictest2518436-21munin-run08:10:000
2496650991816,1cyclictest0-21swapper/011:55:310
2496650991813,3cyclictest886-21NetworkManager07:50:000
2496650991812,5cyclictest2612570-21cat12:35:220
2496650991812,5cyclictest2577231-21cat10:55:180
2496650991812,4cyclictest2608517-21grep12:25:150
2496650991810,6cyclictest2546784-21df09:30:130
249665099180,17cyclictest0-21swapper/012:02:330
249665899177,8cyclictest3056-21PK-Backend09:15:013
2496658991715,1cyclictest3056-21PK-Backend12:32:363
2496658991715,1cyclictest3056-21PK-Backend12:32:353
2496658991715,1cyclictest3056-21PK-Backend07:51:303
249665899171,15cyclictest2572669-21apt-get10:45:033
249665899170,1cyclictest3056-21PK-Backend08:42:343
2496656991715,1cyclictest0-21swapper/212:32:432
2496656991715,1cyclictest0-21swapper/212:32:422
2496656991715,1cyclictest0-21swapper/209:08:492
2496652991715,1cyclictest3056-21PK-Backend11:25:281
2496652991710,6cyclictest436-20systemd-journal08:30:151
2496650991713,3cyclictest2556074-21ntp_offset09:55:190
2496650991710,5cyclictest2544587-21sed09:25:010
2496658991614,1cyclictest3056-21PK-Backend10:40:553
2496658991614,1cyclictest3056-21PK-Backend09:00:383
2496658991613,2cyclictest3056-21PK-Backend08:24:293
2496658991612,3cyclictest2538218-21if_err_enp2s009:05:183
2496658991611,4cyclictest42-21ksoftirqd/308:04:593
249665899160,5cyclictest2517799-21processes08:05:183
249665899160,15cyclictest3056-21PK-Backend07:36:023
249665699169,5cyclictest2569449-21find10:35:102
2496656991612,3cyclictest2567880-21grep10:30:162
2496656991610,4cyclictest2606578-21awk12:20:142
2496656991610,4cyclictest2566620-21grep10:25:172
249665299168,6cyclictest436-20systemd-journal09:30:001
2496652991614,1cyclictest3056-21PK-Backend11:40:161
2496650991614,1cyclictest3056-21PK-Backend09:41:530
249665299150,5cyclictest3056-21PK-Backend11:15:181
2496650991510,4cyclictest2589095-21expr11:30:150
2496650991510,4cyclictest13-21ksoftirqd/008:15:200
249665899149,4cyclictest3056-21PK-Backend08:11:083
2496658991412,1cyclictest3056-21PK-Backend12:18:103
2496658991412,1cyclictest3056-21PK-Backend10:16:163
2496658991411,2cyclictest3056-21PK-Backend09:17:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional