You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-15 - 23:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Sat Aug 15, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
91045621040,3sleep13056-21PK-Backend08:30:011
139099790,69rtkit-daemon0-21swapper/307:09:083
88103627538,13sleep00-21swapper/007:05:500
87991726646,13sleep20-21swapper/207:05:132
88107326241,14sleep10-21swapper/107:06:211
9767392510,3sleep33056-21PK-Backend11:39:313
9459932430,2sleep33056-21PK-Backend10:10:163
9838942310,17sleep013-21ksoftirqd/012:00:010
9428502280,3sleep23056-21PK-Backend10:00:242
9622032270,13sleep126-21ksoftirqd/110:55:221
139099270,4rtkit-daemon1389-21rtkit-daemon10:10:211
881513992316,6cyclictest996275-21cron12:35:002
88151399231,2cyclictest903304-21apt08:10:112
881516992214,6cyclictest898701-21python307:55:193
881513992215,6cyclictest901150-21sed08:05:012
881510992220,1cyclictest0-21swapper/109:00:221
881510992214,2cyclictest26-21ksoftirqd/111:10:001
881510992213,7cyclictest26-21ksoftirqd/107:35:001
881510992211,9cyclictest887152-21unin-run07:25:001
881504992220,1cyclictest0-21swapper/012:36:520
881504992220,1cyclictest0-21swapper/012:25:240
881504992220,1cyclictest0-21swapper/012:25:230
881504992220,1cyclictest0-21swapper/011:21:350
881504992220,1cyclictest0-21swapper/011:09:190
881504992220,1cyclictest0-21swapper/009:29:000
881504992220,1cyclictest0-21swapper/009:13:260
881504992220,1cyclictest0-21swapper/008:27:080
881504992220,1cyclictest0-21swapper/008:19:460
881504992220,1cyclictest0-21swapper/008:10:260
881504992220,1cyclictest0-21swapper/008:02:290
881504992220,1cyclictest0-21swapper/007:59:100
881504992220,1cyclictest0-21swapper/007:37:470
881504992217,3cyclictest8828412sleep007:11:050
88150499221,4cyclictest929572-21munin-run09:24:590
139099220,15rtkit-daemon3056-21PK-Backend08:04:431
139099220,14rtkit-daemon0-21swapper/109:14:081
88151699217,12cyclictest949551-21missed_timers10:20:193
881516992114,5cyclictest436-20systemd-journal10:29:303
88151699211,17cyclictest896972-21python307:50:213
88151699210,20cyclictest3056-21PK-Backend08:52:313
88151699210,20cyclictest0-21swapper/310:18:173
88151699210,19cyclictest3056-21PK-Backend09:12:473
88151699210,19cyclictest3056-21PK-Backend08:46:583
88151699210,19cyclictest3056-21PK-Backend07:25:123
88151399216,4cyclictest141rcu_preempt11:39:412
88151399213,3cyclictest141rcu_preempt07:45:122
88151399212,2cyclictest141rcu_preempt12:15:152
881513992119,1cyclictest0-21swapper/212:27:462
881513992119,1cyclictest0-21swapper/212:27:452
881513992119,1cyclictest0-21swapper/211:00:192
881513992116,4cyclictest896784-21cut07:50:192
881513992116,3cyclictest2085813-21snapd11:23:442
88151399210,5cyclictest0-21swapper/209:20:182
881510992113,6cyclictest26-21ksoftirqd/110:00:201
881510992112,7cyclictest888462-21users07:25:231
881510992112,7cyclictest26-21ksoftirqd/111:55:101
881510992112,7cyclictest26-21ksoftirqd/111:40:121
881510992112,7cyclictest26-21ksoftirqd/110:15:081
881510992112,7cyclictest26-21ksoftirqd/109:05:191
881504992119,1cyclictest0-21swapper/012:22:190
881504992119,1cyclictest0-21swapper/011:39:130
881504992119,1cyclictest0-21swapper/011:31:230
881504992119,1cyclictest0-21swapper/011:28:170
881504992119,1cyclictest0-21swapper/010:43:510
881504992119,1cyclictest0-21swapper/009:52:050
881504992119,1cyclictest0-21swapper/009:37:060
881504992119,1cyclictest0-21swapper/008:55:270
881504992119,1cyclictest0-21swapper/008:32:430
881504992119,1cyclictest0-21swapper/007:40:320
881504992119,1cyclictest0-21swapper/007:40:320
88150499211,18cyclictest907439-21grep08:20:180
88150499210,20cyclictest0-21swapper/012:19:480
88150499210,20cyclictest0-21swapper/011:42:510
88150499210,20cyclictest0-21swapper/009:05:420
88150499210,20cyclictest0-21swapper/008:35:140
88150499210,20cyclictest0-21swapper/008:35:140
139099210,14rtkit-daemon3056-21PK-Backend07:14:381
139099210,14rtkit-daemon0-21swapper/007:49:570
881516992018,1cyclictest0-21swapper/308:02:153
881516992015,3cyclictest920877-21dump-pmu-power09:00:003
881516992014,5cyclictest981398-21ntp_kernel_pll_11:50:193
881516992013,4cyclictest3056-21PK-Backend10:00:143
881516992012,6cyclictest939213-21ntpq09:50:183
881516992011,7cyclictest1-21systemd07:15:013
88151699200,1cyclictest3056-21PK-Backend12:24:223
88151699200,19cyclictest3056-21PK-Backend12:19:363
88151699200,19cyclictest3056-21PK-Backend11:41:103
88151699200,19cyclictest3056-21PK-Backend11:33:203
88151699200,19cyclictest3056-21PK-Backend10:56:213
88151699200,19cyclictest3056-21PK-Backend08:36:233
88151699200,19cyclictest3056-21PK-Backend08:36:223
88151699200,19cyclictest3056-21PK-Backend07:43:373
88151699200,19cyclictest3056-21PK-Backend07:43:373
88151699200,19cyclictest0-21swapper/311:58:353
88151699200,19cyclictest0-21swapper/311:25:283
88151699200,19cyclictest0-21swapper/311:15:413
88151699200,19cyclictest0-21swapper/311:01:423
88151699200,19cyclictest0-21swapper/307:37:353
88151699200,19cyclictest0-21swapper/307:32:553
88151699200,18cyclictest911338-21processes08:30:203
88151399205,8cyclictest436-20systemd-journal12:14:392
88151399204,3cyclictest141rcu_preempt09:05:162
881513992018,1cyclictest0-21swapper/211:52:172
881513992018,1cyclictest0-21swapper/210:39:232
881513992018,1cyclictest0-21swapper/208:50:412
881513992018,1cyclictest0-21swapper/208:25:132
881513992017,2cyclictest0-21swapper/207:15:022
881513992015,4cyclictest978-21snmpd09:49:372
881513992015,3cyclictest978-21snmpd09:52:462
881513992012,2cyclictest34-21ksoftirqd/210:50:222
881513992012,2cyclictest34-21ksoftirqd/210:33:082
881513992012,2cyclictest34-21ksoftirqd/210:10:002
881513992012,2cyclictest34-21ksoftirqd/208:36:282
881513992012,2cyclictest34-21ksoftirqd/208:36:282
88151399200,19cyclictest0-21swapper/211:29:392
88151399200,19cyclictest0-21swapper/211:14:192
88151399200,19cyclictest0-21swapper/210:47:292
88151399200,19cyclictest0-21swapper/210:11:452
88151399200,19cyclictest0-21swapper/209:36:372
88151399200,19cyclictest0-21swapper/209:16:032
88151399200,19cyclictest0-21swapper/208:34:392
88151399200,19cyclictest0-21swapper/207:58:572
88151399200,19cyclictest0-21swapper/207:44:092
88151399200,19cyclictest0-21swapper/207:44:092
881510992015,4cyclictest26-21ksoftirqd/112:05:201
881510992012,7cyclictest26-21ksoftirqd/110:50:241
881510992012,7cyclictest26-21ksoftirqd/109:55:131
881510992012,6cyclictest26-21ksoftirqd/109:25:161
881510992012,2cyclictest26-21ksoftirqd/112:10:091
881510992012,2cyclictest26-21ksoftirqd/107:45:201
88151099201,18cyclictest0-21swapper/108:55:151
881510992011,7cyclictest26-21ksoftirqd/112:20:211
881510992011,7cyclictest26-21ksoftirqd/111:45:131
881510992011,7cyclictest26-21ksoftirqd/110:05:111
881510992011,7cyclictest26-21ksoftirqd/108:40:131
881510992011,7cyclictest26-21ksoftirqd/108:11:081
881510992011,6cyclictest26-21ksoftirqd/112:30:131
88151099200,19cyclictest0-21swapper/109:15:081
881504992018,1cyclictest0-21swapper/012:32:050
881504992018,1cyclictest0-21swapper/012:11:570
881504992018,1cyclictest0-21swapper/012:00:280
881504992018,1cyclictest0-21swapper/011:53:470
881504992018,1cyclictest0-21swapper/011:18:310
881504992018,1cyclictest0-21swapper/011:02:020
881504992018,1cyclictest0-21swapper/010:10:520
881504992018,1cyclictest0-21swapper/010:07:480
881504992018,1cyclictest0-21swapper/009:31:310
881504992018,1cyclictest0-21swapper/007:24:370
881504992013,6cyclictest1-21systemd10:00:000
881504992012,6cyclictest436-20systemd-journal10:05:010
88150499200,19cyclictest0-21swapper/012:08:530
88150499200,19cyclictest0-21swapper/011:46:290
88150499200,19cyclictest0-21swapper/011:12:220
88150499200,19cyclictest0-21swapper/010:50:320
88150499200,19cyclictest0-21swapper/010:46:200
88150499200,19cyclictest0-21swapper/010:22:200
88150499200,19cyclictest0-21swapper/010:19:160
88150499200,19cyclictest0-21swapper/009:42:400
88150499200,19cyclictest0-21swapper/008:40:140
88150499200,19cyclictest0-21swapper/007:50:040
139099200,14rtkit-daemon3056-21PK-Backend09:50:411
139099200,14rtkit-daemon3056-21PK-Backend09:42:171
139099200,14rtkit-daemon3056-21PK-Backend09:34:441
139099200,14rtkit-daemon3056-21PK-Backend07:54:421
881516991917,1cyclictest3056-21PK-Backend09:04:283
881516991917,1cyclictest3056-21PK-Backend07:20:123
881516991917,1cyclictest0-21swapper/310:30:333
881516991917,1cyclictest0-21swapper/307:49:053
881516991916,2cyclictest3056-21PK-Backend09:16:393
881516991915,3cyclictest978-21snmpd11:07:483
881516991912,5cyclictest988175-21cat12:10:143
881516991912,5cyclictest967281-21open_inodes11:10:213
881516991911,6cyclictest935753-21sshd09:40:203
881516991911,6cyclictest934376-21users09:35:233
881516991911,6cyclictest42-21ksoftirqd/308:05:143
88151699190,1cyclictest3056-21PK-Backend10:50:273
88151699190,1cyclictest3056-21PK-Backend10:43:113
88151699190,1cyclictest0-21swapper/311:22:243
88151699190,1cyclictest0-21swapper/308:29:263
88151699190,1cyclictest0-21swapper/308:20:553
88151699190,1cyclictest0-21swapper/308:12:103
88151699190,18cyclictest3056-21PK-Backend07:18:163
881513991917,1cyclictest0-21swapper/212:38:402
881513991917,1cyclictest0-21swapper/212:05:092
881513991917,1cyclictest0-21swapper/211:41:232
881513991917,1cyclictest0-21swapper/211:32:092
881513991917,1cyclictest0-21swapper/210:57:142
881513991917,1cyclictest0-21swapper/210:21:332
881513991917,1cyclictest0-21swapper/209:31:032
881513991917,1cyclictest0-21swapper/209:04:122
881513991917,1cyclictest0-21swapper/207:32:412
881513991917,1cyclictest0-21swapper/207:26:052
881513991914,3cyclictest1095-21runrttasks07:20:172
881513991913,5cyclictest948170-21sed10:15:222
881513991912,6cyclictest34-21ksoftirqd/209:55:202
881513991912,6cyclictest34-21ksoftirqd/209:25:222
881513991912,6cyclictest34-21ksoftirqd/209:10:242
881513991912,5cyclictest991891-21cat12:20:182
881513991912,5cyclictest905170-21grep08:15:142
881513991912,5cyclictest901707-21cut08:05:122
88151399190,7cyclictest881844-21grep07:10:122
88151399190,1cyclictest0-21swapper/208:46:492
88151399190,18cyclictest0-21swapper/211:49:122
88151399190,18cyclictest0-21swapper/209:44:412
88151399190,18cyclictest0-21swapper/208:58:592
88151099194,4cyclictest26-21ksoftirqd/110:45:001
881510991911,6cyclictest26-21ksoftirqd/112:15:221
881510991911,6cyclictest26-21ksoftirqd/112:00:231
881510991911,6cyclictest26-21ksoftirqd/110:45:131
881510991911,6cyclictest26-21ksoftirqd/110:25:211
881510991911,6cyclictest26-21ksoftirqd/109:35:121
881510991911,6cyclictest26-21ksoftirqd/109:22:261
881510991911,6cyclictest26-21ksoftirqd/108:50:211
881510991911,6cyclictest26-21ksoftirqd/107:15:241
881510991911,2cyclictest26-21ksoftirqd/112:35:241
881510991911,2cyclictest26-21ksoftirqd/111:54:311
881510991911,2cyclictest26-21ksoftirqd/111:35:141
881510991911,2cyclictest26-21ksoftirqd/111:20:091
881510991911,2cyclictest26-21ksoftirqd/111:19:081
881510991911,2cyclictest26-21ksoftirqd/111:13:081
881510991911,2cyclictest26-21ksoftirqd/108:23:081
881510991910,2cyclictest26-21ksoftirqd/108:40:001
881510991910,2cyclictest26-21ksoftirqd/108:40:001
881504991917,1cyclictest0-21swapper/009:16:430
881504991911,6cyclictest918335-21needreboot08:50:170
881504991911,6cyclictest889538-21ls07:30:150
88150499190,1cyclictest0-21swapper/010:32:080
88150499190,1cyclictest0-21swapper/009:01:500
88150499190,1cyclictest0-21swapper/008:49:390
88150499190,1cyclictest0-21swapper/008:07:430
139099190,13rtkit-daemon3056-21PK-Backend08:27:461
881516991816,1cyclictest3056-21PK-Backend12:28:363
881516991816,1cyclictest3056-21PK-Backend12:28:363
881516991816,1cyclictest3056-21PK-Backend10:38:243
881516991814,3cyclictest937129-21df09:45:133
881516991811,5cyclictest436-20systemd-journal09:34:543
881516991810,6cyclictest905742-21ntpq08:15:203
88151699180,3cyclictest987633-21awk12:10:003
88151699180,3cyclictest141rcu_preempt11:50:003
881513991816,1cyclictest0-21swapper/210:41:202
881513991816,1cyclictest0-21swapper/210:27:412
881513991812,4cyclictest984370-21mktemp12:00:012
881513991811,2cyclictest34-21ksoftirqd/208:40:212
881513991811,2cyclictest34-21ksoftirqd/207:35:092
881510991815,2cyclictest3056-21PK-Backend10:31:471
881510991811,6cyclictest915745-21awk08:45:121
881510991811,6cyclictest26-21ksoftirqd/107:41:571
881510991811,6cyclictest26-21ksoftirqd/107:41:561
881510991811,5cyclictest26-21ksoftirqd/112:25:201
881510991811,5cyclictest26-21ksoftirqd/112:25:201
881510991811,2cyclictest26-21ksoftirqd/108:06:201
881510991810,6cyclictest26-21ksoftirqd/111:30:211
881510991810,6cyclictest26-21ksoftirqd/111:25:111
881510991810,6cyclictest26-21ksoftirqd/107:55:211
881510991810,2cyclictest26-21ksoftirqd/110:35:161
881510991810,2cyclictest26-21ksoftirqd/109:49:081
881510991810,2cyclictest26-21ksoftirqd/108:15:081
881510991810,2cyclictest26-21ksoftirqd/107:35:271
881504991816,1cyclictest3056-21PK-Backend07:25:580
881504991816,1cyclictest0-21swapper/010:58:240
881504991816,1cyclictest0-21swapper/010:37:420
881504991816,1cyclictest0-21swapper/007:18:150
88150499180,1cyclictest0-21swapper/010:25:240
88151699172,13cyclictest984578-21cpuspeed_turbos12:00:143
88151699171,5cyclictest141rcu_preempt09:55:213
881516991715,1cyclictest3056-21PK-Backend12:33:553
88151699170,1cyclictest3056-21PK-Backend09:25:033
88151399179,2cyclictest34-21ksoftirqd/211:05:112
881513991715,1cyclictest0-21swapper/211:58:592
881513991715,1cyclictest0-21swapper/211:16:152
881510991710,5cyclictest963667-21missed_timers11:00:171
881504991715,1cyclictest0-21swapper/009:47:050
881516991614,1cyclictest3056-21PK-Backend10:08:443
881516991613,2cyclictest3056-21PK-Backend10:45:063
88151699160,5cyclictest141rcu_preempt12:35:183
88151699160,2cyclictest141rcu_preempt09:05:133
88151399169,6cyclictest907233-21cat08:20:172
88151099169,2cyclictest26-21ksoftirqd/110:20:491
88151699145,7cyclictest928681-20journal-offline09:20:163
88151699140,6cyclictest141rcu_preempt08:43:273
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional