You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-15 - 00:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Mon Sep 14, 2026 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
73472183102,29sleep00-21swapper/007:08:090
73292163106,29sleep20-21swapper/207:07:552
72012162105,29sleep30-21swapper/307:06:153
71892161103,29sleep10-21swapper/107:06:041
20902630,2sleep30-21swapper/312:03:453
94852610,2sleep30-21swapper/309:45:003
32372540,2sleep20-21swapper/210:52:012
152772510,5sleep00-21swapper/011:00:250
50092490,4sleep2271rcuc/211:41:082
310512490,1sleep00-21swapper/011:12:410
304592460,1sleep30-21swapper/311:35:563
212082440,1sleep20-21swapper/209:53:222
44032430,2chrt0-21swapper/310:29:233
210992430,1sleep20-21swapper/211:05:172
144252420,1sleep30-21swapper/309:24:303
134652410,4sleep012-21ksoftirqd/011:24:050
70952360,1sleep20-21swapper/210:06:302
158002350,9sleep228-21ksoftirqd/212:37:192
156222350,2sleep20-21swapper/211:50:002
7765993122,7cyclictest0-21swapper/007:28:430
776599291,3cyclictest12494-21grep07:20:150
163772290,2sleep30-21swapper/309:50:023
776699281,25cyclictest0-21swapper/109:01:401
776699281,25cyclictest0-21swapper/108:59:261
776699281,25cyclictest0-21swapper/108:59:261
776699281,25cyclictest0-21swapper/108:44:021
776699281,25cyclictest0-21swapper/108:29:191
776699281,25cyclictest0-21swapper/107:23:561
776599281,25cyclictest0-21swapper/008:29:140
776599281,25cyclictest0-21swapper/007:48:210
174702280,10sleep012-21ksoftirqd/009:25:470
86752270,8sleep334-21ksoftirqd/310:55:433
7768992722,3cyclictest18950irq/143-ahci[0011:20:003
776899271,24cyclictest0-21swapper/308:42:053
776899271,24cyclictest0-21swapper/307:25:123
776799271,24cyclictest0-21swapper/208:46:202
776799271,24cyclictest0-21swapper/207:37:232
776799271,24cyclictest0-21swapper/207:32:282
7766992724,2cyclictest0-21swapper/107:37:551
776699271,24cyclictest0-21swapper/108:45:441
776699271,24cyclictest0-21swapper/108:36:391
776699271,24cyclictest0-21swapper/108:31:131
776699271,24cyclictest0-21swapper/108:20:191
776699271,24cyclictest0-21swapper/108:15:581
776699271,24cyclictest0-21swapper/108:13:451
776699271,24cyclictest0-21swapper/107:54:081
776699271,24cyclictest0-21swapper/107:25:121
7765992724,2cyclictest0-21swapper/007:30:270
776599271,2cyclictest0-21swapper/009:12:120
776599271,25cyclictest0-21swapper/008:38:160
776599271,25cyclictest0-21swapper/008:31:100
776599271,24cyclictest0-21swapper/008:55:440
776599271,24cyclictest0-21swapper/008:55:440
776599271,24cyclictest0-21swapper/008:47:510
776599271,24cyclictest0-21swapper/008:40:090
776599271,24cyclictest0-21swapper/008:16:070
776599271,1cyclictest0-21swapper/007:38:010
246562270,1sleep30-21swapper/311:31:513
188912270,4sleep218896-21processes10:15:212
776899261,24cyclictest0-21swapper/307:47:523
776899261,23cyclictest0-21swapper/308:57:273
776899261,23cyclictest0-21swapper/308:57:263
776899261,23cyclictest0-21swapper/308:47:223
776899261,23cyclictest0-21swapper/308:39:293
776899261,23cyclictest0-21swapper/308:15:563
776899261,23cyclictest0-21swapper/308:09:253
776899261,23cyclictest0-21swapper/308:09:243
776899261,23cyclictest0-21swapper/307:54:263
776899261,23cyclictest0-21swapper/307:20:293
776799261,24cyclictest0-21swapper/208:51:572
776799261,23cyclictest0-21swapper/209:01:432
776799261,23cyclictest0-21swapper/208:57:322
776799261,23cyclictest0-21swapper/208:57:312
776799261,23cyclictest0-21swapper/208:31:472
776799261,23cyclictest0-21swapper/207:54:352
776799261,23cyclictest0-21swapper/207:47:592
776799261,23cyclictest0-21swapper/207:42:412
776799261,23cyclictest0-21swapper/207:22:022
776799260,24cyclictest0-21swapper/208:35:572
7766992623,2cyclictest0-21swapper/107:44:201
776699261,24cyclictest0-21swapper/107:47:561
776699260,24cyclictest0-21swapper/107:30:431
7765992624,1cyclictest0-21swapper/009:01:320
776599261,24cyclictest0-21swapper/008:24:430
776599261,24cyclictest0-21swapper/008:09:200
776599261,24cyclictest0-21swapper/008:09:190
776599261,24cyclictest0-21swapper/008:04:580
776599261,24cyclictest0-21swapper/007:53:510
776599261,1cyclictest0-21swapper/007:43:160
776599261,11cyclictest141rcu_preempt10:40:110
776599260,13cyclictest94862cat09:45:000
69052260,2sleep20-21swapper/208:20:232
776899251,23cyclictest0-21swapper/309:11:183
776899251,23cyclictest0-21swapper/308:20:163
776899251,23cyclictest0-21swapper/308:14:383
776899251,23cyclictest0-21swapper/307:30:073
776899250,23cyclictest0-21swapper/308:31:213
776899250,23cyclictest0-21swapper/308:27:113
776899250,23cyclictest0-21swapper/307:40:073
776899250,23cyclictest0-21swapper/307:36:393
7767992523,1cyclictest0-21swapper/208:07:452
7767992523,1cyclictest0-21swapper/208:07:452
776799251,23cyclictest0-21swapper/208:41:032
776799251,1cyclictest0-21swapper/207:25:182
776799250,23cyclictest0-21swapper/208:26:532
776799250,23cyclictest0-21swapper/208:15:382
7766992523,1cyclictest0-21swapper/108:07:421
7766992523,1cyclictest0-21swapper/108:07:411
776699250,1cyclictest0-21swapper/109:11:371
776599254,11cyclictest4334-21ssh10:05:120
7765992523,1cyclictest0-21swapper/008:10:050
776599250,1cyclictest0-21swapper/008:52:240
286442251,6sleep128660-21ssh12:22:591
155962250,2sleep10-21swapper/109:49:281
7768992422,1cyclictest0-21swapper/307:19:533
7768992416,6cyclictest26569-21ssh09:33:113
776899241,1cyclictest0-21swapper/308:51:073
776799241,1cyclictest0-21swapper/209:08:572
776799240,23cyclictest0-21swapper/208:13:242
776799240,1cyclictest0-21swapper/209:11:382
7768992321,1cyclictest0-21swapper/309:02:543
7768992313,6cyclictest52950irq/141-enp3s0-11:10:113
7767992311,8cyclictest27216-21apt-get07:55:132
7766992321,1cyclictest0-21swapper/108:50:041
226102230,9sleep334-21ksoftirqd/311:55:063
8292220,2sleep122-21ksoftirqd/110:25:481
776899221,9cyclictest164712ssh11:01:333
7768992212,7cyclictest30077-21sort10:00:123
776899221,15cyclictest24237-21ssh12:20:043
776899220,10cyclictest87972sh10:32:023
7767992213,6cyclictest30479-21grep09:35:262
7766992220,1cyclictest0-21swapper/108:04:301
7766992215,5cyclictest31163-21ssh11:12:491
776699220,9cyclictest10402-21kworker/1:011:49:591
776599225,6cyclictest0-21swapper/009:24:570
776599223,4cyclictest0-21swapper/011:54:250
7765992212,5cyclictest18666-21needreboot10:15:180
199332220,2sleep20-21swapper/212:16:132
7768992112,6cyclictest9544-21timerandwakeup11:20:243
776899211,17cyclictest26925-21diskmemload12:11:243
776899210,9cyclictest203232rm11:29:153
776799212,16cyclictest24564-21ssh12:20:132
7767992113,6cyclictest11596-21cat12:10:172
7767992112,6cyclictest7193-21grep10:55:162
7767992112,6cyclictest32577-21grep12:25:242
7767992112,6cyclictest24190-21timerandwakeup11:55:252
7767992111,8cyclictest25289-21date10:45:012
776799210,19cyclictest26244-21sh10:20:312
776699216,13cyclictest3829-21cat09:40:161
776699211,8cyclictest269252diskmemload09:24:151
7766992112,6cyclictest53050irq/142-enp3s0-10:45:121
7766992112,5cyclictest26925-21diskmemload12:00:121
7766992110,8cyclictest16371-21df_inode11:50:141
776599214,2cyclictest141rcu_preempt10:59:270
7765992112,7cyclictest3590-21users09:15:250
7765992111,8cyclictest4939-21ssh11:41:040
7765992111,7cyclictest10077-21basename12:10:000
177242210,2sleep30-21swapper/311:50:253
776899201,5cyclictest20687-21df11:05:133
7768992015,3cyclictest15600-21turbostat11:50:003
7768992013,4cyclictest26925-21diskmemload09:55:123
7768992012,6cyclictest12723-21tune2fs10:35:183
776899201,10cyclictest0-21swapper/310:51:383
7767992013,4cyclictest11160-21sh09:45:222
776699202,13cyclictest32349-21ssh10:50:111
7766992014,4cyclictest29948-21sh10:00:081
7766992013,4cyclictest4697-21sh11:17:031
7766992012,6cyclictest4804-21ntpq12:05:201
7766992012,6cyclictest4070-21date11:40:251
7766992011,7cyclictest22110-21seq09:54:161
7766992011,6cyclictest25967-21open_files09:05:221
776699201,13cyclictest22131-21ssh09:30:131
776599208,7cyclictest52750irq/139-enp3s0-12:10:110
776599207,9cyclictest12182-21ssh12:35:110
776599202,13cyclictest22723-21apt-get11:55:130
776599202,12cyclictest9831-21apt-get09:45:110
7765992012,4cyclictest26925-21diskmemload11:27:510
114252200,3sleep122-21ksoftirqd/110:58:301
776899199,7cyclictest8208-21fschecks_time07:10:163
776899197,8cyclictest25039-21apt-get09:05:133
776899191,6cyclictest0-21swapper/309:37:093
7768991913,4cyclictest24382-21/usr/sbin/munin10:20:223
7768991912,5cyclictest16939-21sed09:25:253
7768991911,6cyclictest4799-21seq09:16:363
7768991911,6cyclictest18676-21cut12:40:013
7768991911,5cyclictest29951-21grep08:00:173
776799197,10cyclictest29201-21ssh11:10:502
776799192,13cyclictest3132-21ssh09:40:112
776799192,12cyclictest23294-21apt-get09:55:142
7767991914,3cyclictest6092-21sh12:30:162
7767991914,3cyclictest31231-21cpu10:25:122
7767991914,3cyclictest16866-21sh10:14:352
7767991913,4cyclictest20038-21sh09:28:222
7767991913,4cyclictest18828-21sh11:03:562
7767991913,4cyclictest14625-21sh10:36:082
7767991913,4cyclictest12743-21sh11:23:222
7767991913,4cyclictest10516-21hwlatdetect07:15:172
7767991912,5cyclictest28667-21grep11:35:172
7767991911,6cyclictest8369-21smartctl07:10:162
7767991911,6cyclictest30291-21ls12:00:172
7767991911,6cyclictest10158-21date09:20:252
776799191,13cyclictest1381-21ssh11:15:142
7767991910,7cyclictest5923-21tune2fs10:30:142
7767991910,7cyclictest22784-21basename09:30:162
776699196,11cyclictest0-21swapper/111:25:141
776699192,12cyclictest10544-21apt-get12:10:041
776699191,8cyclictest31298-21cstates12:25:131
776699191,7cyclictest23672-21diskstats09:55:161
7766991913,4cyclictest17513-21sh09:25:491
7766991913,4cyclictest14585-21sh10:12:171
7766991913,4cyclictest12351-21tr10:35:161
7766991911,6cyclictest28272-21ssh11:35:131
7766991911,6cyclictest22927-21ls11:30:241
7766991911,6cyclictest22822-21seq11:06:121
7766991911,6cyclictest18881-21awk10:40:101
7766991911,6cyclictest16145-21sh11:01:141
7766991911,6cyclictest13138-21seq11:23:451
7766991911,6cyclictest10048-21sh10:09:291
776699191,15cyclictest18952-21kworker/u8:011:55:121
7766991911,5cyclictest12160-21apt-get12:35:111
776699190,18cyclictest0-21swapper/109:35:471
776599199,6cyclictest52750irq/139-enp3s0-12:00:120
776599198,2cyclictest12-21ksoftirqd/010:25:230
776599192,5cyclictest0-21swapper/010:34:260
776599192,12cyclictest24154-21kworker/u8:010:00:110
7765991913,3cyclictest21987-21sh09:30:060
7765991912,5cyclictest22560-21uname11:30:190
7765991911,6cyclictest8589-21sed07:10:180
7765991910,7cyclictest560-21dbus-daemon10:40:000
7765991910,6cyclictest278-21systemd-journal11:40:010
7765991910,6cyclictest24523-21grep12:20:130
318152190,2sleep00-21swapper/010:49:540
776899187,9cyclictest0-21swapper/312:06:393
776899182,7cyclictest17697-21apt-get10:15:113
776899181,6cyclictest3424-21ls11:40:193
776899181,5cyclictest0-21swapper/312:31:553
776899181,3cyclictest141rcu_preempt12:25:123
7768991813,3cyclictest4717-21fschecks_count10:05:163
7768991812,3cyclictest26925-21diskmemload12:19:423
7768991810,6cyclictest29165-21seq10:47:143
7768991810,6cyclictest11154-21sensors_temp09:45:213
776799182,11cyclictest29374-21apt-get08:00:132
7767991813,3cyclictest5795-21sh12:05:442
7767991813,3cyclictest30908-21missed_timers10:00:182
7767991813,3cyclictest18313-21sh11:27:132
7767991813,3cyclictest16069-21cpu11:50:112
7767991812,4cyclictest5961-21sh09:17:462
7767991811,5cyclictest26238-21expr10:45:142
7767991810,6cyclictest22881-21sed11:30:232
776699187,3cyclictest22-21ksoftirqd/110:24:321
776699182,11cyclictest7806-21apt-get07:10:131
7766991813,3cyclictest7896-21sh12:31:111
7766991813,3cyclictest5509-21sh09:17:181
7766991812,4cyclictest23703-21sh12:20:001
7766991812,3cyclictest7957-21sh10:31:111
7766991811,5cyclictest18437-21irqstats10:15:171
7766991810,6cyclictest28913-21seq07:57:471
776599182,11cyclictest32328-21apt-get10:50:110
776599181,8cyclictest28740-21ssh09:35:080
776599181,4cyclictest0-21swapper/009:54:390
7765991813,3cyclictest1345-21/usr/sbin/munin11:15:210
7765991812,4cyclictest12-21ksoftirqd/009:10:010
7765991812,3cyclictest24382-21/usr/sbin/munin10:20:180
7765991811,4cyclictest9386-21grep11:45:120
7765991811,4cyclictest7880-21sh12:31:100
7765991810,6cyclictest21168-21seq12:17:270
7768991712,3cyclictest27190-21/usr/sbin/munin07:55:203
7768991712,3cyclictest21318-21sh10:41:053
776899171,12cyclictest17300-21munin-run10:15:003
776699171,11cyclictest9953-21apt-get07:15:121
776599179,5cyclictest27907-21latency07:55:170
7765991712,3cyclictest20293-21/usr/sbin/munin11:05:250
7765991711,4cyclictest26925-21diskmemload12:25:060
7765991711,3cyclictest26925-21diskmemload09:55:320
7765991711,3cyclictest10815-21sh10:10:020
7765991612,2cyclictest9927-21/usr/sbin/munin07:15:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional