You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-08 - 21:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Apr 08, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
338653422230,6sleep20-21swapper/207:05:232
341693221220,3sleep00-21swapper/008:30:190
345698121050,3sleep00-21swapper/010:25:190
345859621040,4sleep03056-21PK-Backend10:30:170
338702026845,16sleep30-21swapper/307:07:583
338666126746,14sleep00-21swapper/007:05:250
338688526140,14sleep10-21swapper/107:06:161
139099510,10rtkit-daemon3056-21PK-Backend09:25:013
35047862490,4sleep00-21swapper/012:40:000
33904602450,4sleep10-21swapper/107:15:181
34095732380,3chrt3056-21PK-Backend08:10:173
139099380,4rtkit-daemon0-21swapper/208:50:312
33881602330,4sleep10-21swapper/107:10:171
3387324992514,9cyclictest3423956-21timerandwakeup08:50:203
338732499235,16cyclictest3446589-21pluginstate09:55:183
3387318992317,4cyclictest3486800-21grep11:50:001
3387318992316,5cyclictest3406861-21cron08:05:001
338731499232,4cyclictest3392938-21sh07:25:010
139099230,15rtkit-daemon3056-21PK-Backend09:14:193
34468102220,4sleep13056-21PK-Backend09:55:211
3387322992220,1cyclictest0-21swapper/211:58:522
3387322992220,1cyclictest0-21swapper/208:46:132
3387322992220,1cyclictest0-21swapper/208:20:242
338732299220,21cyclictest3392583-21chrt07:20:362
338732299220,20cyclictest0-21swapper/211:17:012
3387318992215,5cyclictest26-21ksoftirqd/111:15:151
3387318992214,6cyclictest3472135-21awk11:05:211
3387318992213,7cyclictest3470205-21users11:00:211
338731899220,20cyclictest3491270-21ntp_states12:00:151
3387314992220,1cyclictest0-21swapper/012:25:290
3387314992220,1cyclictest0-21swapper/012:00:340
3387314992220,1cyclictest0-21swapper/009:05:060
3387314992220,1cyclictest0-21swapper/007:18:410
139099220,15rtkit-daemon3056-21PK-Backend09:27:353
139099220,15rtkit-daemon3056-21PK-Backend09:18:123
139099220,14rtkit-daemon0-21swapper/312:15:593
139099220,14rtkit-daemon0-21swapper/312:06:153
3387324992119,1cyclictest3056-21PK-Backend08:07:003
3387324992119,1cyclictest0-21swapper/312:20:363
3387324992119,1cyclictest0-21swapper/311:22:223
3387324992119,1cyclictest0-21swapper/310:55:043
3387324992119,1cyclictest0-21swapper/309:54:293
3387324992119,1cyclictest0-21swapper/309:42:393
3387324992119,1cyclictest0-21swapper/308:27:223
3387324992118,2cyclictest3056-21PK-Backend11:41:083
338732499211,7cyclictest3462490-21/usr/sbin/munin10:40:213
3387324992113,6cyclictest3419260-21cron08:39:593
338732499210,19cyclictest3056-21PK-Backend11:32:163
338732499210,19cyclictest3056-21PK-Backend11:32:163
338732499210,19cyclictest3056-21PK-Backend07:44:163
338732499210,19cyclictest3056-21PK-Backend07:26:123
3387322992119,1cyclictest0-21swapper/212:20:342
3387322992119,1cyclictest0-21swapper/211:51:072
3387322992119,1cyclictest0-21swapper/211:22:422
3387322992119,1cyclictest0-21swapper/210:27:542
3387322992119,1cyclictest0-21swapper/210:14:002
3387322992119,1cyclictest0-21swapper/209:56:312
3387322992119,1cyclictest0-21swapper/209:50:292
3387322992119,1cyclictest0-21swapper/207:32:122
338732299210,20cyclictest0-21swapper/211:32:022
338732299210,20cyclictest0-21swapper/211:32:012
338732299210,20cyclictest0-21swapper/209:41:022
338732299210,1cyclictest0-21swapper/211:44:122
3387318992113,6cyclictest3399332-21python307:40:211
3387314992119,1cyclictest0-21swapper/012:21:030
3387314992119,1cyclictest0-21swapper/011:56:430
3387314992119,1cyclictest0-21swapper/011:52:510
3387314992119,1cyclictest0-21swapper/011:38:230
3387314992119,1cyclictest0-21swapper/010:12:260
3387314992119,1cyclictest0-21swapper/010:09:490
3387314992119,1cyclictest0-21swapper/009:16:160
3387314992119,1cyclictest0-21swapper/009:11:210
3387314992119,1cyclictest0-21swapper/008:16:000
3387314992119,1cyclictest0-21swapper/007:46:260
3387314992118,2cyclictest3056-21PK-Backend10:15:030
3387314992117,3cyclictest0-21swapper/008:25:120
3387314992117,3cyclictest0-21swapper/007:40:170
3387314992116,3cyclictest3486810-21munin-run11:49:590
338731499210,20cyclictest0-21swapper/012:32:310
338731499210,20cyclictest0-21swapper/011:11:410
338731499210,20cyclictest0-21swapper/009:37:320
338731499210,20cyclictest0-21swapper/009:37:310
338731499210,20cyclictest0-21swapper/008:24:240
338731499210,19cyclictest3056-21PK-Backend09:50:090
139099210,15rtkit-daemon0-21swapper/310:18:113
139099210,14rtkit-daemon0-21swapper/310:30:243
139099210,14rtkit-daemon0-21swapper/308:00:383
139099210,14rtkit-daemon0-21swapper/307:39:523
139099210,14rtkit-daemon0-21swapper/208:18:182
139099210,14rtkit-daemon0-21swapper/207:47:522
34629402200,3sleep00-21swapper/010:45:010
3387324992018,1cyclictest3056-21PK-Backend11:36:433
3387324992018,1cyclictest3056-21PK-Backend11:08:513
3387324992018,1cyclictest3056-21PK-Backend10:03:543
3387324992018,1cyclictest0-21swapper/310:54:043
3387324992018,1cyclictest0-21swapper/309:46:043
3387324992018,1cyclictest0-21swapper/307:59:243
3387324992012,6cyclictest3416514-21cat08:30:153
338732499200,1cyclictest3056-21PK-Backend08:43:453
338732499200,1cyclictest0-21swapper/307:49:373
338732499200,19cyclictest3056-21PK-Backend11:03:223
338732499200,19cyclictest3056-21PK-Backend10:46:343
3387322992018,1cyclictest0-21swapper/210:56:062
3387322992018,1cyclictest0-21swapper/210:21:522
3387322992018,1cyclictest0-21swapper/210:00:572
3387322992018,1cyclictest0-21swapper/209:31:022
3387322992018,1cyclictest0-21swapper/209:18:032
3387322992018,1cyclictest0-21swapper/208:56:212
3387322992018,1cyclictest0-21swapper/207:39:432
338732299200,1cyclictest0-21swapper/207:28:272
338732299200,19cyclictest0-21swapper/212:15:072
338732299200,19cyclictest0-21swapper/211:46:152
338732299200,19cyclictest0-21swapper/211:36:422
338732299200,19cyclictest0-21swapper/211:25:322
338732299200,19cyclictest0-21swapper/210:52:142
338732299200,19cyclictest0-21swapper/210:38:002
338732299200,19cyclictest0-21swapper/210:30:302
338732299200,19cyclictest0-21swapper/210:16:382
338732299200,19cyclictest0-21swapper/210:06:102
338732299200,19cyclictest0-21swapper/209:47:522
338732299200,19cyclictest0-21swapper/209:13:452
338732299200,19cyclictest0-21swapper/208:41:542
338732299200,19cyclictest0-21swapper/208:04:032
338731899204,14cyclictest3426199-21cron08:59:591
3387318992018,1cyclictest3056-21PK-Backend11:25:491
3387318992018,1cyclictest3056-21PK-Backend10:35:131
3387318992018,1cyclictest3056-21PK-Backend10:13:091
3387318992018,1cyclictest0-21swapper/111:31:161
3387318992018,1cyclictest0-21swapper/111:31:151
3387318992018,1cyclictest0-21swapper/111:20:211
3387318992018,1cyclictest0-21swapper/110:29:451
3387318992014,5cyclictest26-21ksoftirqd/108:05:181
3387318992012,6cyclictest3409241-21df_inode08:10:131
338731899200,1cyclictest0-21swapper/107:56:471
338731899200,19cyclictest3056-21PK-Backend12:16:521
338731899200,19cyclictest3056-21PK-Backend12:05:571
338731899200,19cyclictest3056-21PK-Backend10:41:271
338731899200,19cyclictest3056-21PK-Backend09:54:291
338731899200,19cyclictest0-21swapper/111:57:251
338731899200,19cyclictest0-21swapper/110:21:201
338731899200,19cyclictest0-21swapper/110:05:321
338731899200,19cyclictest0-21swapper/110:02:551
338731899200,19cyclictest0-21swapper/107:39:451
338731899200,19cyclictest0-21swapper/107:27:201
338731899200,18cyclictest436-20systemd-journal11:43:281
338731899200,18cyclictest3056-21PK-Backend08:47:181
3387314992018,1cyclictest0-21swapper/012:19:140
3387314992018,1cyclictest0-21swapper/012:13:460
3387314992018,1cyclictest0-21swapper/011:32:570
3387314992018,1cyclictest0-21swapper/011:32:570
3387314992018,1cyclictest0-21swapper/011:24:380
3387314992018,1cyclictest0-21swapper/010:58:290
3387314992018,1cyclictest0-21swapper/008:45:400
3387314992018,1cyclictest0-21swapper/007:12:530
338731499200,19cyclictest0-21swapper/008:55:340
139099200,3rtkit-daemon0-21swapper/208:06:452
139099200,14rtkit-daemon0-21swapper/307:17:593
139099200,14rtkit-daemon0-21swapper/209:00:012
338732499192,13cyclictest3503206-21apt12:35:113
3387324991917,1cyclictest3056-21PK-Backend11:49:263
3387324991917,1cyclictest3056-21PK-Backend10:08:073
3387324991917,1cyclictest0-21swapper/312:25:023
3387324991917,1cyclictest0-21swapper/312:11:293
3387324991917,1cyclictest0-21swapper/310:36:263
338732499190,1cyclictest3056-21PK-Backend12:02:113
338732499190,1cyclictest3056-21PK-Backend09:37:383
338732499190,1cyclictest3056-21PK-Backend09:37:383
338732499190,1cyclictest0-21swapper/308:55:483
338732499190,18cyclictest3056-21PK-Backend11:12:293
338732499190,18cyclictest3056-21PK-Backend10:24:573
3387322991917,1cyclictest0-21swapper/212:12:172
3387322991917,1cyclictest0-21swapper/211:14:112
3387322991917,1cyclictest0-21swapper/210:48:222
3387322991917,1cyclictest0-21swapper/209:05:202
3387322991917,1cyclictest0-21swapper/208:37:222
3387322991917,1cyclictest0-21swapper/207:13:262
3387322991914,3cyclictest1095-21runrttasks08:30:122
3387322991913,4cyclictest3461558-21grep10:40:112
3387322991911,6cyclictest3469862-21ntpq11:00:172
338732299190,1cyclictest0-21swapper/212:32:042
338732299190,1cyclictest0-21swapper/212:25:012
338732299190,1cyclictest0-21swapper/208:29:042
338732299190,1cyclictest0-21swapper/207:16:162
338732299190,18cyclictest0-21swapper/212:38:202
338732299190,18cyclictest0-21swapper/208:12:342
338732299190,18cyclictest0-21swapper/207:40:302
3387318991917,1cyclictest3056-21PK-Backend09:49:421
3387318991917,1cyclictest3056-21PK-Backend08:19:061
3387318991917,1cyclictest0-21swapper/112:35:311
3387318991917,1cyclictest0-21swapper/108:51:091
3387318991917,1cyclictest0-21swapper/107:46:411
3387318991914,3cyclictest3433869-21if_enp2s009:20:151
3387318991912,5cyclictest3481865-21awk11:35:101
3387318991912,5cyclictest3432407-21ntpq09:15:171
3387318991910,7cyclictest3391836-21df07:20:121
338731899190,1cyclictest0-21swapper/107:31:401
338731899190,18cyclictest3056-21PK-Backend10:57:031
338731899190,18cyclictest0-21swapper/111:51:451
338731899190,18cyclictest0-21swapper/109:39:571
338731899190,18cyclictest0-21swapper/109:39:571
338731899190,18cyclictest0-21swapper/108:26:081
338731899190,18cyclictest0-21swapper/108:20:271
338731899190,17cyclictest3056-21PK-Backend12:32:531
3387314991917,1cyclictest3056-21PK-Backend10:48:560
3387314991917,1cyclictest3056-21PK-Backend08:35:250
3387314991917,1cyclictest0-21swapper/011:08:510
3387314991917,1cyclictest0-21swapper/010:52:470
3387314991917,1cyclictest0-21swapper/010:03:000
3387314991917,1cyclictest0-21swapper/009:56:570
3387314991917,1cyclictest0-21swapper/008:54:120
3387314991917,1cyclictest0-21swapper/008:10:320
3387314991917,1cyclictest0-21swapper/008:07:340
3387314991917,1cyclictest0-21swapper/007:59:580
3387314991917,1cyclictest0-21swapper/007:31:240
3387314991912,4cyclictest3485071-21idleruntime-cro11:44:590
3387314991911,6cyclictest13-21ksoftirqd/011:01:320
338731499190,1cyclictest0-21swapper/008:41:210
338731499190,1cyclictest0-21swapper/007:38:420
139099190,12rtkit-daemon0-21swapper/207:58:542
3387324991816,1cyclictest3056-21PK-Backend11:16:083
3387324991816,1cyclictest3056-21PK-Backend07:20:443
3387324991815,2cyclictest3056-21PK-Backend10:29:563
3387324991811,6cyclictest3422272-21sed08:45:213
338732499181,15cyclictest3489664-21python311:55:203
338732499180,17cyclictest3056-21PK-Backend09:09:133
338732499180,17cyclictest0-21swapper/312:32:533
338732499180,16cyclictest3478768-21df_inode11:25:133
3387322991816,1cyclictest0-21swapper/211:05:382
3387322991814,3cyclictest1095-21runrttasks07:52:542
3387322991810,6cyclictest3493256-21ntpq12:05:182
3387318991816,1cyclictest3056-21PK-Backend10:53:241
3387318991816,1cyclictest3056-21PK-Backend10:33:231
3387318991816,1cyclictest3056-21PK-Backend08:40:281
3387318991816,1cyclictest3056-21PK-Backend08:37:311
3387318991816,1cyclictest0-21swapper/109:05:091
3387318991811,5cyclictest3437439-21ls09:30:151
3387318991811,5cyclictest3426878-21which09:00:141
3387314991816,1cyclictest3056-21PK-Backend09:41:440
3387314991816,1cyclictest0-21swapper/010:21:520
3387314991816,1cyclictest0-21swapper/009:01:010
3387314991812,5cyclictest3436095-21find09:25:210
338731499181,16cyclictest3493995-21perf12:10:000
3387314991810,6cyclictest3393576-21df_inode07:25:130
3387324991715,1cyclictest3056-21PK-Backend11:54:543
3387324991715,1cyclictest0-21swapper/310:12:203
3387324991712,3cyclictest3401914-21/usr/sbin/munin07:50:133
338732499170,15cyclictest3412579-21/usr/sbin/munin08:20:103
3387322991715,1cyclictest0-21swapper/209:36:502
3387322991715,1cyclictest0-21swapper/209:36:502
3387322991715,1cyclictest0-21swapper/209:24:262
3387322991711,4cyclictest3492004-21apt-get12:05:012
338731899172,13cyclictest3463555-21tr10:45:111
3387318991715,1cyclictest3056-21PK-Backend12:24:291
3387318991715,1cyclictest3056-21PK-Backend12:13:141
3387318991714,2cyclictest3056-21PK-Backend08:31:501
3387318991711,4cyclictest3473604-21grep11:10:171
3387314991715,1cyclictest0-21swapper/009:45:090
3387314991715,1cyclictest0-21swapper/008:04:100
3387314991712,4cyclictest918-21rs:main0
3387314991712,3cyclictest3437548-21grep09:30:160
3387314991711,4cyclictest3474655-21acpi11:15:090
3387324991614,1cyclictest3056-21PK-Backend09:01:163
3387322991613,2cyclictest3436616-21sh09:30:002
338731899169,5cyclictest3435222-21cat09:25:111
338731899169,5cyclictest3430301-21cat09:10:121
338731899167,8cyclictest3056-21PK-Backend07:54:591
3387318991614,1cyclictest3056-21PK-Backend12:28:411
338731499168,6cyclictest3434888-21perf09:25:010
3387314991614,1cyclictest0-21swapper/010:38:490
139099160,3rtkit-daemon3056-21PK-Backend07:33:373
3387324991513,1cyclictest3056-21PK-Backend07:13:473
338732499150,13cyclictest3410626-21gpg-connect-age08:15:013
3387318991510,4cyclictest3440943-21grep09:40:171
3387314991510,3cyclictest3402456-21grep07:50:150
3387318991410,3cyclictest3056-21PK-Backend10:16:011
139099120,4rtkit-daemon3056-21PK-Backend09:30:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional