You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-22 - 23:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Sun Mar 22, 2026 12:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
219415122980,20sleep30-21swapper/307:05:123
224967121390,3sleep10-21swapper/109:40:001
230982721260,3sleep33056-21PK-Backend12:30:113
228675121000,3sleep02286750-21wc11:25:010
219525026840,22sleep00-21swapper/007:06:100
219549426746,14sleep20-21swapper/207:09:182
219538426141,14sleep10-21swapper/107:07:561
22949052490,3sleep30-21swapper/311:45:213
22415412450,1sleep00-21swapper/009:15:180
22615522440,3sleep33056-21PK-Backend10:10:213
22932342390,1sleep12293236-21awk11:40:221
2195686993514,5cyclictest2206815-21cron07:39:591
2195686993113,5cyclictest1095-21runrttasks09:20:011
2195686992913,3cyclictest2280429-21if_enp2s011:05:161
2195683992510,4cyclictest13-21ksoftirqd/011:20:010
23062572240,11sleep013-21ksoftirqd/012:20:110
2195692992415,7cyclictest2254275-21users09:50:223
219568399246,10cyclictest13-21ksoftirqd/007:55:000
2195688992316,5cyclictest2212014-21cron07:55:002
2195683992315,7cyclictest2213772-21munin-run08:00:000
139099230,15rtkit-daemon0-21swapper/007:14:390
22994292220,2sleep33056-21PK-Backend12:00:123
2195692992219,2cyclictest3056-21PK-Backend09:17:133
2195688992210,9cyclictest883-21dbus-daemon08:40:032
2195686992215,5cyclictest2311210-21cron12:35:011
2195686992214,6cyclictest2258993-21ls10:05:121
219568699220,20cyclictest2269058-21runrttasks10:31:421
219568399228,4cyclictest13-21ksoftirqd/007:35:190
2195683992220,1cyclictest0-21swapper/012:04:500
2195683992220,1cyclictest0-21swapper/009:00:310
2195683992220,1cyclictest0-21swapper/008:26:040
2195683992219,1cyclictest161rcuc/011:43:380
139099220,14rtkit-daemon0-21swapper/008:21:420
139099220,13rtkit-daemon0-21swapper/007:28:160
2195692992119,1cyclictest3056-21PK-Backend11:57:283
2195692992119,1cyclictest3056-21PK-Backend11:27:133
2195692992119,1cyclictest3056-21PK-Backend10:55:223
2195692992119,1cyclictest3056-21PK-Backend09:39:253
2195692992119,1cyclictest3056-21PK-Backend09:28:503
2195692992119,1cyclictest3056-21PK-Backend09:08:563
2195692992119,1cyclictest3056-21PK-Backend07:45:063
2195692992119,1cyclictest0-21swapper/312:08:353
2195692992119,1cyclictest0-21swapper/311:20:173
2195692992119,1cyclictest0-21swapper/307:30:003
2195692992119,1cyclictest0-21swapper/307:14:593
2195692992118,2cyclictest3056-21PK-Backend07:20:543
2195692992118,1cyclictest3056-21PK-Backend11:35:093
219569299210,20cyclictest3056-21PK-Backend11:12:193
219569299210,20cyclictest3056-21PK-Backend11:06:233
219569299210,20cyclictest3056-21PK-Backend10:42:513
219569299210,20cyclictest3056-21PK-Backend08:21:243
219569299210,20cyclictest0-21swapper/308:10:353
219569299210,19cyclictest3056-21PK-Backend12:27:363
219569299210,19cyclictest3056-21PK-Backend12:15:533
219569299210,19cyclictest3056-21PK-Backend09:48:573
219569299210,19cyclictest3056-21PK-Backend09:10:443
2195688992119,1cyclictest0-21swapper/210:53:092
2195688992113,6cyclictest2290996-21memory11:35:172
219568899210,20cyclictest0-21swapper/211:54:272
219568899210,20cyclictest0-21swapper/211:42:022
219568899210,20cyclictest0-21swapper/210:58:302
219568899210,20cyclictest0-21swapper/209:47:462
219568899210,20cyclictest0-21swapper/208:27:222
219568899210,19cyclictest0-21swapper/212:30:232
219568899210,19cyclictest0-21swapper/207:45:452
2195686992119,1cyclictest0-21swapper/112:23:431
2195686992119,1cyclictest0-21swapper/112:16:331
2195686992119,1cyclictest0-21swapper/110:49:551
2195686992119,1cyclictest0-21swapper/107:10:131
2195686992113,6cyclictest2300153-21seq12:00:221
2195686992113,6cyclictest2254693-21http09:55:001
219568699210,20cyclictest0-21swapper/111:23:201
219568399216,4cyclictest13-21ksoftirqd/012:10:120
219568399212,9cyclictest13-21ksoftirqd/010:15:000
219568399212,5cyclictest13-21ksoftirqd/010:35:010
2195683992119,1cyclictest0-21swapper/012:27:410
2195683992119,1cyclictest0-21swapper/011:27:050
2195683992119,1cyclictest0-21swapper/010:37:230
2195683992119,1cyclictest0-21swapper/010:30:010
2195683992119,1cyclictest0-21swapper/010:20:480
2195683992119,1cyclictest0-21swapper/010:15:410
2195683992119,1cyclictest0-21swapper/009:47:490
2195683992119,1cyclictest0-21swapper/009:41:060
2195683992119,1cyclictest0-21swapper/009:32:210
2195683992119,1cyclictest0-21swapper/009:28:290
2195683992119,1cyclictest0-21swapper/009:20:450
2195683992119,1cyclictest0-21swapper/009:10:240
2195683992119,1cyclictest0-21swapper/008:56:530
2195683992119,1cyclictest0-21swapper/008:32:530
219568399210,20cyclictest0-21swapper/011:38:420
219568399210,20cyclictest0-21swapper/010:54:000
219568399210,20cyclictest0-21swapper/008:35:300
139099210,14rtkit-daemon0-21swapper/009:05:070
139099210,13rtkit-daemon0-21swapper/108:55:311
2195692992018,1cyclictest3056-21PK-Backend12:14:043
2195692992018,1cyclictest3056-21PK-Backend09:40:263
2195692992018,1cyclictest3056-21PK-Backend08:32:263
2195692992017,2cyclictest3056-21PK-Backend08:47:193
2195692992011,7cyclictest2233610-21cron08:55:003
219569299200,19cyclictest3056-21PK-Backend12:23:573
219569299200,19cyclictest3056-21PK-Backend08:57:133
219569299200,19cyclictest3056-21PK-Backend08:17:593
219569299200,19cyclictest0-21swapper/307:31:363
2195688992018,1cyclictest0-21swapper/210:31:332
2195688992018,1cyclictest0-21swapper/210:23:552
2195688992018,1cyclictest0-21swapper/208:09:172
2195688992018,1cyclictest0-21swapper/207:20:162
2195688992017,2cyclictest3056-21PK-Backend08:52:362
219568899200,6cyclictest22415402ntp_offset09:15:182
219568899200,1cyclictest0-21swapper/212:29:212
219568899200,1cyclictest0-21swapper/210:40:112
219568899200,19cyclictest0-21swapper/210:29:162
219568899200,19cyclictest0-21swapper/210:15:312
219568899200,19cyclictest0-21swapper/210:10:102
219568899200,19cyclictest0-21swapper/209:30:492
219568899200,19cyclictest0-21swapper/209:09:132
219568899200,19cyclictest0-21swapper/209:00:552
219568899200,19cyclictest0-21swapper/208:16:072
219568899200,19cyclictest0-21swapper/207:12:462
2195686992018,1cyclictest0-21swapper/112:11:001
2195686992018,1cyclictest0-21swapper/108:36:591
2195686992018,1cyclictest0-21swapper/108:07:521
2195686992018,1cyclictest0-21swapper/107:46:431
2195686992015,3cyclictest2244019-21cut09:25:011
2195686992014,4cyclictest2156-21gnome-shell09:10:091
2195686992011,7cyclictest2233598-21cron08:55:001
219568699200,19cyclictest0-21swapper/110:28:041
219568399203,6cyclictest13-21ksoftirqd/007:15:200
2195683992018,1cyclictest0-21swapper/011:00:210
2195683992018,1cyclictest0-21swapper/007:21:210
2195683992016,3cyclictest161rcuc/009:50:410
219568399201,10cyclictest1788597-21WebKitNetworkPr09:58:120
2195683992010,2cyclictest141rcu_preempt10:00:050
219568399200,19cyclictest3056-21PK-Backend08:40:430
219568399200,19cyclictest0-21swapper/011:50:520
219568399200,19cyclictest0-21swapper/011:49:500
219568399200,19cyclictest0-21swapper/009:39:040
219568399200,19cyclictest0-21swapper/008:06:500
219568399200,19cyclictest0-21swapper/007:30:270
2195692991917,1cyclictest3056-21PK-Backend11:52:483
2195692991917,1cyclictest3056-21PK-Backend07:15:463
2195692991917,1cyclictest0-21swapper/308:41:033
2195692991916,2cyclictest3056-21PK-Backend11:01:303
2195692991916,2cyclictest3056-21PK-Backend10:16:013
2195692991916,2cyclictest3056-21PK-Backend10:05:473
2195692991911,6cyclictest2256265-21awk09:55:203
2195692991911,6cyclictest2236311-21awk09:00:173
219569299190,1cyclictest3056-21PK-Backend10:30:213
219569299190,1cyclictest3056-21PK-Backend08:04:003
219569299190,1cyclictest3056-21PK-Backend07:59:133
219569299190,1cyclictest0-21swapper/312:35:403
2195688991917,1cyclictest0-21swapper/212:07:032
2195688991917,1cyclictest0-21swapper/208:12:422
2195688991917,1cyclictest0-21swapper/207:25:362
2195688991910,7cyclictest2285571-21df_inode11:20:132
219568899190,1cyclictest3056-21PK-Backend11:55:282
219568899190,1cyclictest0-21swapper/210:35:522
219568899190,1cyclictest0-21swapper/208:49:462
219568899190,1cyclictest0-21swapper/208:37:022
219568899190,1cyclictest0-21swapper/207:40:452
219568899190,1cyclictest0-21swapper/207:18:412
219568899190,18cyclictest0-21swapper/209:27:442
219568899190,18cyclictest0-21swapper/209:12:032
219568699192,12cyclictest2300538-21apt-get12:05:041
2195686991917,1cyclictest0-21swapper/111:10:301
2195686991917,1cyclictest0-21swapper/109:28:501
2195686991914,3cyclictest2297078-21latency_hist11:55:011
2195686991912,5cyclictest26-21ksoftirqd/112:35:211
2195686991912,5cyclictest2308562-21cat12:25:171
2195686991912,5cyclictest2291869-21cron11:40:001
2195686991912,5cyclictest2271052-21cron10:40:001
219568699190,1cyclictest0-21swapper/108:32:111
219568699190,1cyclictest0-21swapper/108:21:171
219568699190,18cyclictest0-21swapper/108:16:441
219568699190,18cyclictest0-21swapper/107:42:581
219568399192,6cyclictest13-21ksoftirqd/011:55:140
219568399191,7cyclictest13-21ksoftirqd/012:30:180
2195683991917,1cyclictest0-21swapper/008:53:140
2195683991917,1cyclictest0-21swapper/008:19:140
2195683991917,1cyclictest0-21swapper/008:14:130
2195683991917,1cyclictest0-21swapper/008:00:280
2195683991915,3cyclictest161rcuc/012:20:010
2195683991911,6cyclictest2280760-21awk11:05:200
219568399190,1cyclictest0-21swapper/012:35:470
219568399190,1cyclictest0-21swapper/007:45:350
219568399190,18cyclictest0-21swapper/012:07:420
219568399190,18cyclictest0-21swapper/011:12:120
139099190,12rtkit-daemon0-21swapper/008:45:460
2195692991816,1cyclictest3056-21PK-Backend11:34:083
2195692991816,1cyclictest3056-21PK-Backend07:35:343
2195692991816,1cyclictest0-21swapper/311:43:283
2195692991815,2cyclictest3056-21PK-Backend08:29:013
2195692991811,4cyclictest2284948-21idleruntime-cro11:20:013
2195692991810,7cyclictest2211308-21ntp_states07:50:203
2195688991816,1cyclictest0-21swapper/212:35:012
2195688991816,1cyclictest0-21swapper/210:48:492
2195688991812,5cyclictest2251191-21cron09:45:002
2195688991811,5cyclictest2282271-21sed11:10:162
219568699189,8cyclictest918-21rs:main1
219568699189,7cyclictest2258275-21users10:00:201
219568699189,7cyclictest2235612-21apt09:00:101
219568699189,7cyclictest2231487-21users08:45:241
219568699187,9cyclictest2238841-21unin-run09:10:001
219568699182,10cyclictest883-21dbus-daemon09:30:031
2195686991816,1cyclictest0-21swapper/110:17:241
2195686991812,5cyclictest2211367-21cut07:50:201
2195686991811,5cyclictest436-20systemd-journal10:21:561
2195686991810,6cyclictest883-21dbus-daemon08:00:011
2195686991810,6cyclictest2294622-21ntpq11:45:171
2195686991810,5cyclictest436-20systemd-journal11:30:001
219568699180,1cyclictest0-21swapper/107:16:491
219568399183,4cyclictest13-21ksoftirqd/007:40:110
2195683991816,1cyclictest0-21swapper/011:34:010
2195692991715,1cyclictest3056-21PK-Backend10:29:203
2195692991715,1cyclictest3056-21PK-Backend10:20:073
2195692991715,1cyclictest3056-21PK-Backend08:35:033
2195692991713,3cyclictest2257204-21apt10:00:103
2195692991713,3cyclictest2243401-21python309:20:193
219568899179,6cyclictest2255987-21ps09:55:172
219568899179,6cyclictest2221933-21awk08:20:172
2195688991715,1cyclictest0-21swapper/211:48:442
2195688991712,3cyclictest2225077-21tr08:30:122
2195688991711,5cyclictest2260006-21awk10:05:212
2195688991711,4cyclictest436-20systemd-journal07:39:552
2195688991711,4cyclictest2281455-21cron11:09:592
2195688991710,5cyclictest436-20systemd-journal11:05:002
2195688991710,5cyclictest34-21ksoftirqd/207:30:022
2195688991710,5cyclictest2299657-21cut12:00:172
219568699179,6cyclictest2214010-21find08:00:101
2195686991712,3cyclictest2218422-21needreboot08:10:171
2195686991711,5cyclictest2278885-21ls11:00:161
2195686991711,4cyclictest2202251-21grep07:25:191
2195686991711,4cyclictest2200475-21cat07:20:161
2195686991710,5cyclictest2224605-21readlink08:30:011
219568399178,7cyclictest2276997-21cat10:55:140
219568399175,4cyclictest13-21ksoftirqd/010:48:210
2195683991712,4cyclictest13-21ksoftirqd/010:45:010
219568399170,5cyclictest1213-21kerneloops10:05:450
2195692991614,1cyclictest3056-21PK-Backend10:50:153
2195692991613,2cyclictest3056-21PK-Backend10:37:443
219568899169,5cyclictest2254675-21apt-get09:54:592
2195688991614,1cyclictest3056-21PK-Backend08:55:272
2195688991614,1cyclictest0-21swapper/208:04:172
2195688991612,3cyclictest2213757-21dump-pmu-power07:59:592
2195688991611,3cyclictest2284219-21open_files11:15:192
219568699168,6cyclictest2288685-21apt11:30:121
219568699162,9cyclictest2271336-21apt10:40:121
219568699162,10cyclictest82332-21update-manager09:58:141
2195686991614,1cyclictest0-21swapper/110:52:441
2195686991612,3cyclictest2283446-21acpi11:15:101
2195686991612,3cyclictest2203765-21tr07:30:111
219568699160,15cyclictest0-21swapper/110:11:561
219568699160,14cyclictest2276748-21expr10:55:121
2195692991513,1cyclictest3056-21PK-Backend09:30:533
2195692991513,1cyclictest3056-21PK-Backend08:09:483
219568899159,4cyclictest2248679-21cut09:35:172
2195688991512,2cyclictest2197595-21munin-node11:25:132
219568899151,11cyclictest2257207-21chrt10:00:112
2195688991510,4cyclictest2288953-21expr11:30:132
2195688991510,3cyclictest2303213-21idleruntime12:10:152
2195686991511,3cyclictest1095-21runrttasks09:40:141
139099150,3rtkit-daemon0-21swapper/108:44:231
219568899149,4cyclictest2305144-21sed12:15:172
219568899147,5cyclictest2243311-21open_inodes09:20:172
2195688991412,1cyclictest0-21swapper/212:22:052
219568699149,3cyclictest2251455-21acpi09:45:111
219569299138,4cyclictest3056-21PK-Backend10:48:233
219569299129,2cyclictest3056-21PK-Backend07:40:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional