You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-08 - 05:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Wed Apr 08, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1691399270,1cyclictest0-21swapper/321:46:083
1691399250,1cyclictest611-21irqbalance22:27:453
16909992523,1cyclictest0-21swapper/223:31:582
1690999250,1cyclictest313-21systemd-journal20:25:002
1690799250,24cyclictest10547-21diskmemload23:38:061
1690799250,1cyclictest0-21swapper/122:03:081
1690799250,1cyclictest0-21swapper/100:09:111
1690399250,24cyclictest21298-21cron20:25:000
1690399250,24cyclictest0-21swapper/023:12:060
16913992424,0cyclictest0-21swapper/321:35:103
16913992422,1cyclictest0-21swapper/322:59:043
16913992422,1cyclictest0-21swapper/300:39:033
1691399240,23cyclictest0-21swapper/323:35:263
1691399240,23cyclictest0-21swapper/300:08:513
1691399240,0cyclictest0-21swapper/321:18:163
1691399240,0cyclictest0-21swapper/300:19:043
16909992423,1cyclictest0-21swapper/223:26:122
16909992423,0cyclictest10547-21diskmemload22:24:222
16909992423,0cyclictest0-21swapper/200:09:172
1690999240,23cyclictest21689-21awk19:20:002
1690999240,23cyclictest10547-21diskmemload22:38:012
1690999240,23cyclictest10547-21diskmemload21:36:462
16907992424,0cyclictest0-21swapper/123:59:251
16907992423,0cyclictest10547-21diskmemload21:26:351
1690799240,23cyclictest10547-21diskmemload23:15:311
1690799240,23cyclictest0-21swapper/122:42:001
1690799240,1cyclictest10547-21diskmemload21:22:331
1690799240,0cyclictest0-21swapper/123:53:451
1690799240,0cyclictest0-21swapper/123:53:451
1690799240,0cyclictest0-21swapper/123:14:401
16903992423,1cyclictest10547-21diskmemload23:18:210
16903992423,1cyclictest0-21swapper/023:34:300
16903992423,1cyclictest0-21swapper/022:36:060
16903992423,1cyclictest0-21swapper/021:29:050
16903992421,2cyclictest27276-21rm23:00:500
16903992421,2cyclictest0-21swapper/023:20:550
1690399240,24cyclictest10547-21diskmemload21:44:200
1690399240,24cyclictest0-21swapper/023:27:370
1690399240,24cyclictest0-21swapper/022:30:330
1690399240,23cyclictest31796-21ssh22:02:420
1690399240,23cyclictest22437-21apt-get21:54:020
1690399240,0cyclictest10547-21diskmemload00:34:130
16913992323,0cyclictest0-21swapper/322:05:383
16913992321,1cyclictest10547-21diskmemload23:19:583
16913992321,1cyclictest0-21swapper/322:50:043
16913992321,1cyclictest0-21swapper/322:35:043
16913992321,1cyclictest0-21swapper/300:14:453
16913992320,2cyclictest9517-21diskmemload23:46:383
16913992320,2cyclictest32747-21sh23:06:173
16913992320,2cyclictest32427-21sh21:31:413
16913992320,2cyclictest25719-21sh21:56:083
1691399231,2cyclictest10130-21processes22:44:093
1691399230,23cyclictest0-21swapper/322:33:113
1691399230,23cyclictest0-21swapper/321:44:213
1691399230,22cyclictest24459-21sh00:00:423
1691399230,22cyclictest20405-21sh21:50:473
1691399230,22cyclictest11709-21perf20:05:003
1691399230,22cyclictest10547-21diskmemload21:11:483
1691399230,0cyclictest0-21swapper/323:58:403
1691399230,0cyclictest0-21swapper/321:29:233
16909992323,0cyclictest0-21swapper/223:43:022
16909992323,0cyclictest0-21swapper/221:51:302
16909992323,0cyclictest0-21swapper/200:28:342
16909992322,1cyclictest10547-21diskmemload00:19:142
16909992321,1cyclictest0-21swapper/223:58:412
16909992321,1cyclictest0-21swapper/220:35:012
16909992321,1cyclictest0-21swapper/219:55:012
1690999230,23cyclictest0-21swapper/223:35:162
1690999230,22cyclictest29665-21ssh22:31:502
1690999230,22cyclictest10547-21diskmemload22:01:022
1690999230,22cyclictest0-21swapper/223:19:042
1690999230,22cyclictest0-21swapper/223:12:292
1690999230,22cyclictest0-21swapper/223:00:502
1690999230,1cyclictest3167-21ntp_states23:09:082
1690999230,1cyclictest10547-21diskmemload21:41:002
1690999230,1cyclictest0-21swapper/222:57:462
1690999230,0cyclictest10547-21diskmemload22:26:522
1690999230,0cyclictest0-21swapper/221:29:052
16907992323,0cyclictest0-21swapper/123:46:541
16907992323,0cyclictest0-21swapper/121:47:561
16907992323,0cyclictest0-21swapper/100:03:171
16907992321,1cyclictest31763-21rm21:30:421
16907992321,1cyclictest0-21swapper/122:15:091
16907992320,2cyclictest5757-21ssh23:43:461
1690799231,21cyclictest21319-21turbostat20:25:001
1690799230,2cyclictest1259-21ssh23:07:491
1690799230,23cyclictest27230-21kworker/u8:123:05:011
1690799230,22cyclictest5002-21ssh21:36:331
1690799230,22cyclictest29270-21idleruntime-cro19:35:001
1690799230,22cyclictest27929-21sh00:35:131
1690799230,22cyclictest22703-21sh22:56:351
1690799230,22cyclictest22401-21/usr/sbin/munin21:54:111
1690799230,22cyclictest12822-21ssh21:13:091
1690799230,22cyclictest12120-21ssh00:20:061
1690799230,22cyclictest0-21swapper/122:11:071
1690799230,22cyclictest0-21swapper/121:17:041
1690799230,0cyclictest10547-21diskmemload21:58:011
1690799230,0cyclictest0-21swapper/100:12:011
16903992323,0cyclictest0-21swapper/023:35:040
16903992323,0cyclictest0-21swapper/022:26:520
16903992323,0cyclictest0-21swapper/022:11:450
16903992322,1cyclictest1080-21ssh00:09:170
16903992320,2cyclictest14111-21sh00:23:120
1690399231,22cyclictest13226-21ssh21:13:530
1690399230,2cyclictest18418-21ssh22:20:160
1690399230,23cyclictest10547-21diskmemload22:08:420
1690399230,22cyclictest28108-21sh00:35:290
1690399230,22cyclictest21675-21munin-run19:20:000
1690399230,22cyclictest19086-21ssh00:28:030
1690399230,22cyclictest13256-21ssh22:47:560
1690399230,22cyclictest10761-21ssh23:48:350
1690399230,22cyclictest0-21swapper/023:43:160
1690399230,1cyclictest24917-21ssh00:01:330
1690399230,1cyclictest10547-21diskmemload22:18:200
1690399230,0cyclictest0-21swapper/021:36:380
1690399230,0cyclictest0-21swapper/021:22:450
16913992222,0cyclictest10547-21diskmemload23:28:493
16913992222,0cyclictest0-21swapper/323:40:073
16913992222,0cyclictest0-21swapper/323:24:293
16913992221,1cyclictest10547-21diskmemload22:20:103
16913992220,1cyclictest0-21swapper/323:50:283
16913992220,1cyclictest0-21swapper/323:50:283
1691399220,21cyclictest10547-21diskmemload23:30:123
1691399220,21cyclictest0-21swapper/322:45:173
1691399220,21cyclictest0-21swapper/322:13:403
1691399220,21cyclictest0-21swapper/319:54:033
1691399220,21cyclictest0-21swapper/300:20:053
1691399220,1cyclictest0-21swapper/323:12:233
1691399220,1cyclictest0-21swapper/322:18:343
1691399220,1cyclictest0-21swapper/322:01:183
1691399220,1cyclictest0-21swapper/300:30:013
1691399220,0cyclictest10547-21diskmemload00:25:123
16909992222,0cyclictest0-21swapper/223:50:432
16909992222,0cyclictest0-21swapper/223:50:432
16909992222,0cyclictest0-21swapper/222:46:332
16909992222,0cyclictest0-21swapper/222:42:532
16909992222,0cyclictest0-21swapper/221:55:322
16909992222,0cyclictest0-21swapper/221:23:452
16909992222,0cyclictest0-21swapper/200:22:012
16909992222,0cyclictest0-21swapper/200:02:322
16909992221,1cyclictest0-21swapper/200:30:252
16909992220,2cyclictest3330-21ssh22:05:462
16909992220,1cyclictest18236-21sh21:18:322
16909992220,1cyclictest0-21swapper/221:33:172
1690999220,21cyclictest32050-21perf19:40:012
1690999220,21cyclictest17898-21df23:24:032
1690999220,21cyclictest10547-21diskmemload23:45:242
1690999220,21cyclictest10547-21diskmemload22:16:142
1690999220,21cyclictest0-21swapper/221:14:042
1690999220,21cyclictest0-21swapper/200:35:392
1690999220,21cyclictest0-21swapper/200:13:062
1690999220,1cyclictest8004-21ssh22:10:112
1690999220,1cyclictest0-21swapper/222:53:442
16907992221,1cyclictest6985-21ssh00:15:051
16907992220,1cyclictest3320-21basename20:55:011
16907992220,1cyclictest24067-21turbostat.cron19:25:001
1690799220,21cyclictest24094-21ssh22:26:071
1690799220,21cyclictest0-21swapper/123:30:201
1690799220,21cyclictest0-21swapper/123:20:271
1690799220,1cyclictest19217-21sh22:21:341
1690799220,1cyclictest18768-21ssh00:27:341
1690799220,1cyclictest17028-21ssh22:50:401
1690799220,1cyclictest1392-21sh22:35:181
1690799220,1cyclictest0-21swapper/121:41:471
16903992222,0cyclictest0-21swapper/021:56:460
16903992222,0cyclictest0-21swapper/021:33:510
16903992220,1cyclictest29253-21latency_hist19:35:000
1690399220,22cyclictest17763-21ssh22:51:590
1690399220,22cyclictest0-21swapper/023:55:310
1690399220,21cyclictest8684-21apt-get21:09:020
1690399220,21cyclictest2502-21sh00:11:070
1690399220,21cyclictest10547-21diskmemload21:45:030
1690399220,1cyclictest6828-21diskmemload22:40:480
1690399220,1cyclictest14663-21ssh23:51:410
1690399220,1cyclictest14663-21ssh23:51:410
1690399220,0cyclictest0-21swapper/022:58:200
16913992121,0cyclictest0-21swapper/321:20:153
16913992120,1cyclictest21651-21apt19:20:003
16913992119,1cyclictest0-21swapper/321:05:003
1691399210,20cyclictest26127-21awk20:35:013
1691399210,20cyclictest22243-21apt-get19:24:033
1691399210,20cyclictest0-21swapper/319:49:033
1691399210,1cyclictest14105-21apt20:10:003
1691399210,0cyclictest0-21swapper/319:14:023
16909992121,0cyclictest0-21swapper/219:34:022
16909992121,0cyclictest0-21swapper/219:14:022
16907992121,0cyclictest10547-21diskmemload22:45:401
16907992121,0cyclictest0-21swapper/123:25:061
16907992120,1cyclictest0-21swapper/120:14:021
16907992120,0cyclictest19828-21apt-get19:19:031
1690799210,21cyclictest0-21swapper/119:49:031
1690799210,20cyclictest0-21swapper/121:04:031
1690799210,0cyclictest10547-21diskmemload22:05:381
1690799210,0cyclictest0-21swapper/122:30:311
16903992121,0cyclictest5024-21apt-get19:54:030
16903992121,0cyclictest0-21swapper/020:59:030
16903992119,1cyclictest0-21swapper/020:35:010
1690399210,21cyclictest0-21swapper/021:15:590
1690399210,21cyclictest0-21swapper/000:15:200
1690399210,20cyclictest0-21swapper/020:39:030
16913992020,0cyclictest0-21swapper/320:29:023
1691399200,20cyclictest0-21swapper/319:35:023
1691399200,0cyclictest0-21swapper/319:34:023
16909992020,0cyclictest0-21swapper/221:09:012
16909992020,0cyclictest0-21swapper/219:44:012
1690999200,0cyclictest0-21swapper/220:29:022
16907992020,0cyclictest0-21swapper/120:44:021
16907992020,0cyclictest0-21swapper/119:39:021
16907992019,0cyclictest9833-21apt-get20:04:031
1690799200,0cyclictest0-21swapper/119:44:021
16903992019,1cyclictest4432-21idleruntime-cro19:50:010
16903992019,1cyclictest32641-21apt-get19:44:010
16903992019,1cyclictest0-21swapper/020:29:020
16903992019,1cyclictest0-21swapper/019:14:020
16913991919,0cyclictest0-21swapper/321:09:013
16913991919,0cyclictest0-21swapper/320:54:023
16913991918,1cyclictest0-21swapper/320:24:023
16913991917,1cyclictest24689-21apt-get19:25:023
16909991919,0cyclictest0-21swapper/220:54:022
16909991919,0cyclictest0-21swapper/220:19:042
16909991919,0cyclictest0-21swapper/219:49:032
16909991919,0cyclictest0-21swapper/219:24:032
16909991918,1cyclictest908-21perf20:50:012
16909991918,1cyclictest6293-21apt-get21:04:032
16907991919,0cyclictest0-21swapper/120:09:031
16907991919,0cyclictest0-21swapper/119:59:021
16907991916,2cyclictest23728-21munin-run20:30:011
16903991919,0cyclictest30937-21perf20:45:010
16903991918,1cyclictest1468-21apt-get20:54:020
16913991818,0cyclictest17080-21apt-get20:19:043
16913991818,0cyclictest0-21swapper/320:59:033
16909991818,0cyclictest12281-21apt-get20:09:032
16909991817,1cyclictest3875-21apt-get20:59:032
16907991818,0cyclictest0-21swapper/120:59:031
16907991816,1cyclictest18936-21sendmail-msp20:20:001
16903991818,0cyclictest0-21swapper/019:35:020
16913991715,1cyclictest0-21swapper/320:39:033
16907991715,1cyclictest606-21dbus-daemon20:50:011
16907991715,1cyclictest26682-21apt-get20:39:021
16903991717,0cyclictest0-21swapper/020:19:040
16903991717,0cyclictest0-21swapper/019:24:030
16903991716,1cyclictest0-21swapper/020:50:010
16913991615,0cyclictest0-21swapper/319:59:023
16909991614,1cyclictest0-21swapper/220:14:022
16903991616,0cyclictest0-21swapper/021:04:030
16903991616,0cyclictest0-21swapper/020:09:030
1691399153,11cyclictest793-21lldpd20:14:103
1691399150,14cyclictest2066-21perf19:45:013
16909991512,2cyclictest10496-21tail20:04:052
16903991514,1cyclictest14675-21apt-get20:14:030
16913991414,0cyclictest0-21swapper/320:45:283
1691399140,0cyclictest0-21swapper/320:41:303
16909991414,0cyclictest0-21swapper/220:37:262
16909991414,0cyclictest0-21swapper/219:25:362
1690999140,14cyclictest0-21swapper/219:55:092
16907991414,0cyclictest0-21swapper/119:50:191
1690799140,14cyclictest0-21swapper/120:30:151
1690799140,0cyclictest0-21swapper/119:25:031
1690799140,0cyclictest0-21swapper/119:10:361
16903991414,0cyclictest0-21swapper/020:00:110
16903991414,0cyclictest0-21swapper/019:25:150
1690399140,0cyclictest0-21swapper/019:56:100
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional