You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-10 - 15:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Mon Aug 10, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
323892238163,24sleep20-21swapper/207:05:312
327192220192,19sleep10-21swapper/107:09:491
324922204168,24sleep30-21swapper/307:06:543
324992201163,26sleep00-21swapper/007:06:580
2465021820,4sleep236299cyclictest09:15:182
36299104100,3cyclictest0-21swapper/208:56:222
362999692,3cyclictest0-21swapper/208:00:272
362997572,2cyclictest0-21swapper/211:33:212
46572700,1sleep00-21swapper/012:10:180
250972410,2sleep00-21swapper/009:15:240
35899406,2cyclictest23-21ksoftirqd/108:07:031
362993835,2cyclictest0-21swapper/207:48:062
35899371,11cyclictest10758-21head09:55:281
35899361,7cyclictest1922-21tail09:35:251
358993514,3cyclictest20840-21gltestperf10:20:151
358993512,3cyclictest28353-21/usr/sbin/munin09:25:151
358993510,3cyclictest11674-21/usr/sbin/munin08:45:181
35899342,4cyclictest0-21swapper/109:50:191
358993411,4cyclictest12959-21/usr/sbin/munin11:15:181
35899340,17cyclictest0-21swapper/111:12:011
36999331,13cyclictest0-21swapper/309:07:363
35899338,3cyclictest0-21swapper/108:14:561
35899332,4cyclictest6417-21df_inode12:15:111
35899332,17cyclictest16484-40dnf12:35:421
35899331,26cyclictest0-21swapper/112:09:401
310512330,3sleep011-21rcuc/010:45:010
30442330,5sleep135899cyclictest08:25:201
35899322,7cyclictest11050-21perl12:25:161
35899321,4cyclictest23733-21sh09:15:021
35899321,3cyclictest9157-21latency12:20:191
358993213,4cyclictest32204-21sort12:00:161
35899321,25cyclictest0-21swapper/110:26:441
35899321,22cyclictest0-21swapper/109:15:531
35899321,10cyclictest27276-21ls11:50:011
358993210,3cyclictest23339-21/usr/sbin/munin08:00:161
35899311,4cyclictest31451-21cat10:45:131
35899311,3cyclictest6763-21awk09:50:011
35899311,15cyclictest0-21swapper/110:39:591
358993111,3cyclictest3747-21proc_pri09:40:221
35899311,13cyclictest0-21swapper/109:20:051
35899311,11cyclictest0-21swapper/112:13:321
36999301,22cyclictest0-21swapper/309:28:563
36999300,23cyclictest0-21swapper/312:10:003
35899309,3cyclictest827-21runrttasks09:06:101
35899306,2cyclictest0-21swapper/108:35:121
35899304,4cyclictest14254-21uptime12:30:271
358993011,3cyclictest9173-21grep08:40:161
358993011,3cyclictest8840-21cat07:25:201
358993011,3cyclictest30817-21sort08:15:181
358993011,3cyclictest14538-21/usr/sbin/munin07:40:121
35899301,13cyclictest0-21swapper/111:22:521
35899301,13cyclictest0-21swapper/110:09:531
35899301,13cyclictest0-21swapper/109:04:291
35899301,13cyclictest0-21swapper/107:19:161
35899301,12cyclictest0-21swapper/108:55:081
35899301,11cyclictest0-21swapper/111:00:041
358993010,3cyclictest4348-21awk10:55:221
358993010,3cyclictest2633-21latency_hist07:15:011
35899300,9cyclictest0-21swapper/107:55:131
35899300,7cyclictest0-21swapper/111:05:221
35299301,14cyclictest8906-21kworker/u8:210:10:010
36999291,13cyclictest0-21swapper/308:20:073
36999291,13cyclictest0-21swapper/307:56:113
36999291,11cyclictest0-21swapper/311:59:053
36999290,16cyclictest0-21swapper/310:00:403
35899295,8cyclictest1137-21cpuspeed_turbos10:50:131
35899292,3cyclictest27524-21cpuspeed_turbos11:50:121
35899291,9cyclictest22654-21taskset11:38:041
35899291,7cyclictest30553-21/usr/sbin/munin09:30:261
35899291,26cyclictest0-21swapper/110:10:581
358992912,3cyclictest12682-21sh07:35:141
35899291,16cyclictest0-21swapper/108:23:111
35899291,14cyclictest0-21swapper/107:31:281
358992911,3cyclictest12056-21hddtemp_smartct10:00:171
35899291,11cyclictest0-21swapper/108:31:121
358992910,3cyclictest25680-21ntpq10:30:221
36999281,22cyclictest0-21swapper/309:37:363
36999281,17cyclictest0-21swapper/307:31:023
36999281,13cyclictest0-21swapper/308:07:153
36999281,11cyclictest0-21swapper/311:32:433
36999281,11cyclictest0-21swapper/310:43:363
36999281,10cyclictest0-21swapper/307:15:163
36999280,22cyclictest0-21swapper/311:35:313
36999280,21cyclictest0-21swapper/312:10:083
36999280,16cyclictest0-21swapper/309:22:043
36999280,13cyclictest0-21swapper/311:50:393
36999280,12cyclictest0-21swapper/309:32:063
35899281,13cyclictest0-21swapper/110:41:481
358992810,3cyclictest30897-21smart_sda11:55:251
35299288,16cyclictest4578-21awk09:45:010
35299282,9cyclictest14137-21kworker/0:212:36:150
36999271,12cyclictest0-21swapper/308:31:183
36999271,11cyclictest0-21swapper/308:26:523
36999271,10cyclictest0-21swapper/309:44:523
36999271,10cyclictest0-21swapper/308:47:543
36999270,18cyclictest0-21swapper/308:01:133
36999270,14cyclictest0-21swapper/311:14:013
36999270,13cyclictest0-21swapper/310:37:303
36999270,12cyclictest0-21swapper/312:15:143
36999270,12cyclictest0-21swapper/310:21:403
36999270,12cyclictest0-21swapper/310:11:503
36999270,12cyclictest0-21swapper/310:09:583
36999270,11cyclictest0-21swapper/307:36:153
36299277,8cyclictest21532-21diskmemload10:35:262
358992712,2cyclictest0-21swapper/107:49:301
35899271,12cyclictest19144-21cstates07:50:121
35899271,12cyclictest19144-21cstates07:50:121
35899271,12cyclictest0-21swapper/107:24:451
35899270,16cyclictest0-21swapper/111:42:521
35299272,9cyclictest10496-21kworker/0:109:04:510
35299272,8cyclictest30115-21kworker/0:209:32:540
35299272,10cyclictest6714-21kworker/0:009:51:380
35299271,9cyclictest12699-21kworker/0:010:26:220
35299271,12cyclictest13379-21perf10:05:000
36999261,9cyclictest0-21swapper/311:04:433
36999261,9cyclictest0-21swapper/309:46:103
36999261,13cyclictest0-21swapper/310:19:163
36999261,13cyclictest0-21swapper/307:53:413
36999261,13cyclictest0-21swapper/307:53:413
36999261,10cyclictest21532-21diskmemload11:47:413
36999261,10cyclictest0-21swapper/312:01:553
36999261,10cyclictest0-21swapper/308:40:283
36999260,11cyclictest0-21swapper/310:31:463
36999260,11cyclictest0-21swapper/309:18:143
35899260,3cyclictest0-21swapper/111:26:251
35899260,3cyclictest0-21swapper/111:26:251
35299262,8cyclictest2978-21kworker/0:111:01:300
35299261,8cyclictest14137-21kworker/0:212:32:510
35299261,8cyclictest10496-21kworker/0:108:58:270
36999251,9cyclictest0-21swapper/307:47:173
36999251,11cyclictest0-21swapper/312:38:403
36999251,11cyclictest0-21swapper/309:12:303
36999251,11cyclictest0-21swapper/308:14:093
36999251,10cyclictest0-21swapper/312:33:103
36999251,10cyclictest0-21swapper/311:23:163
36999250,2cyclictest0-21swapper/310:25:363
36999250,15cyclictest0-21swapper/311:07:173
36999250,14cyclictest0-21swapper/310:59:413
36999250,12cyclictest0-21swapper/308:17:333
36299259,4cyclictest0-21swapper/211:10:242
36299259,3cyclictest0-21swapper/210:54:292
35899258,2cyclictest0-21swapper/111:30:131
35299256,2cyclictest1372-21kworker/0:308:21:510
35299252,8cyclictest2978-21kworker/0:110:57:380
35299251,8cyclictest32369-21kworker/0:309:59:340
35299251,8cyclictest11495-21kworker/0:111:36:460
352992513,10cyclictest7443-21perf11:09:550
35299251,10cyclictest10177-21kworker/0:012:29:030
36999248,3cyclictest0-21swapper/311:25:073
36999248,3cyclictest0-21swapper/311:25:063
36999247,4cyclictest9345-21ntp_states12:20:223
36999246,10cyclictest0-21swapper/308:50:523
36999241,9cyclictest0-21swapper/308:56:543
36999241,7cyclictest0-21swapper/307:21:143
36999241,10cyclictest0-21swapper/312:27:503
36999241,10cyclictest0-21swapper/307:44:033
36999241,10cyclictest0-21swapper/307:26:123
369992410,3cyclictest0-21swapper/308:35:383
36999240,11cyclictest0-21swapper/311:43:153
36999240,11cyclictest0-21swapper/310:53:123
36999240,11cyclictest0-21swapper/309:57:283
35899241,10cyclictest0-21swapper/110:17:481
35299249,2cyclictest26854-21kworker/0:211:45:460
35299242,9cyclictest7935-21kworker/0:112:20:070
35299242,9cyclictest21760-21kworker/0:009:14:230
35299242,8cyclictest1398-21kworker/0:312:18:310
35299241,9cyclictest13927-21kworker/0:207:24:080
36999237,3cyclictest0-21swapper/307:11:023
36999231,7cyclictest0-21swapper/311:17:443
362992315,5cyclictest26591-21munin-run10:35:022
362992312,6cyclictest827-21runrttasks07:34:152
35899231,11cyclictest0-21swapper/108:51:441
35299231,6cyclictest0-21swapper/010:19:480
36999227,3cyclictest0-21swapper/309:05:003
36999221,11cyclictest0-21swapper/309:53:423
36999220,10cyclictest0-21swapper/310:46:003
36299221,4cyclictest0-21swapper/209:29:042
35299229,3cyclictest12699-21kworker/0:010:34:180
35299229,2cyclictest14587-21kworker/0:008:52:210
35299222,8cyclictest18284-21kworker/0:011:30:300
35299222,8cyclictest11495-21kworker/0:111:16:220
35299222,11cyclictest32358-21kworker/0:109:37:020
35299221,8cyclictest12699-21kworker/0:010:12:140
36299216,6cyclictest0-21swapper/210:49:532
36299211,5cyclictest0-21swapper/211:47:532
362992114,4cyclictest8098-21ls09:50:242
35299219,2cyclictest32369-21kworker/0:309:45:100
35299219,2cyclictest27476-21kworker/0:208:18:170
35299214,2cyclictest13927-21kworker/0:207:13:020
35299211,9cyclictest5084-21kworker/0:311:11:300
35299211,9cyclictest21833-21kworker/0:310:24:360
35299211,9cyclictest16017-21kworker/0:211:25:340
35299211,9cyclictest16017-21kworker/0:211:25:340
35299211,9cyclictest11914-21kworker/0:208:01:360
35299211,9cyclictest10496-21kworker/0:108:49:410
35299211,8cyclictest7971-21kworker/0:008:41:270
35299211,6cyclictest29163-21irqstats09:25:170
35299211,12cyclictest26817-21/usr/sbin/munin10:35:260
161472211,2sleep00-21swapper/011:24:150
36299201,8cyclictest30784-21cat08:15:182
362992015,3cyclictest4590-21grep08:30:152
362992010,2cyclictest30-21rcuc/210:40:272
35299209,2cyclictest25216-21kworker/0:008:06:500
35299205,13cyclictest14314-21munin-run07:40:000
35299202,9cyclictest3891-21kworker/0:108:30:350
35299202,9cyclictest24087-21kworker/0:110:47:460
35299202,8cyclictest31439-21kworker/0:212:01:270
35299201,16cyclictest3468-21/usr/sbin/munin08:25:240
35299201,11cyclictest26156-21/usr/sbin/munin09:20:240
35299200,8cyclictest647-21kworker/0:210:51:140
36299195,3cyclictest80712wc12:20:002
36299192,6cyclictest23736-21munin-run09:15:012
36299191,7cyclictest0-21swapper/211:00:172
36299191,6cyclictest0-21swapper/211:50:552
36299191,5cyclictest0-21swapper/210:15:102
362991914,3cyclictest21061-21grep09:05:252
36299191,12cyclictest0-21swapper/211:40:052
35299192,8cyclictest20894-21kworker/0:007:57:040
35299191,9cyclictest1398-21kworker/0:312:06:350
35299191,4cyclictest19283-21df_inode07:50:130
35299191,4cyclictest19283-21df_inode07:50:130
35299191,15cyclictest30232-21grep11:55:170
35299191,10cyclictest16733-21/usr/sbin/munin07:45:210
36299185,2cyclictest0-21swapper/207:26:572
36299182,5cyclictest5750irq/9-acpi11:15:122
36299181,7cyclictest0-21swapper/212:25:172
36299181,6cyclictest0-21swapper/210:13:202
36299181,5cyclictest0-21swapper/211:55:262
36299181,5cyclictest0-21swapper/211:05:142
36299181,4cyclictest0-21swapper/209:40:152
362991811,4cyclictest22461-40dnf11:35:362
362991811,4cyclictest13269-21grep07:35:192
36299181,10cyclictest0-21swapper/210:28:092
362991810,5cyclictest0-21swapper/207:50:242
362991810,5cyclictest0-21swapper/207:50:242
35299187,3cyclictest808-21snmpd07:32:290
35299181,6cyclictest0-21swapper/011:53:110
352991812,3cyclictest20185-21gltestperf09:05:160
352991811,4cyclictest909-21sssd_nss08:10:230
36299176,3cyclictest87350irq/126-enp1s0-12:20:142
36299171,7cyclictest5416-21sed12:10:262
36299171,7cyclictest3557-21/usr/sbin/munin07:15:222
36299171,6cyclictest0-21swapper/212:36:402
36299171,5cyclictest0-21swapper/210:20:242
36299171,4cyclictest0-21swapper/210:56:402
36299171,3cyclictest0-21swapper/212:06:372
362991712,3cyclictest6293-21unixbench_multi09:45:252
36299171,15cyclictest0-21swapper/207:23:382
36299171,15cyclictest0-21swapper/207:13:332
36299171,14cyclictest0-21swapper/207:55:162
362991711,3cyclictest6986-21grep08:35:162
362991711,3cyclictest17538-21cstates09:00:132
362991711,3cyclictest11718-21df10:00:152
36299171,11cyclictest0-21swapper/212:32:272
362991710,4cyclictest0-21swapper/208:05:162
35299176,2cyclictest11495-21kworker/0:111:40:380
35299174,2cyclictest14133-21kworker/0:107:40:140
352991712,3cyclictest3029-21ntp_states07:15:040
36299165,3cyclictest12050irq/123-ahci[0008:50:002
36299164,3cyclictest26914-21if_enp2s009:20:162
36299161,6cyclictest12961-21acpi08:50:132
36299161,6cyclictest0-21swapper/211:20:102
36299161,5cyclictest27875-21tr08:10:122
36299161,5cyclictest0-21swapper/209:55:152
36299161,4cyclictest0-21swapper/209:30:022
362991612,3cyclictest0-21swapper/212:02:322
36299161,14cyclictest0-21swapper/210:05:522
362991611,3cyclictest32750-21/usr/sbin/munin09:35:222
362991611,3cyclictest32473-21diskstats08:20:142
36299161,12cyclictest16862-21grep11:25:152
36299161,12cyclictest16862-21grep11:25:142
36299161,12cyclictest10119-21grep08:40:262
97212151,2sleep00-21swapper/007:27:370
36299158,4cyclictest14960-21fschecks_count07:40:142
36299149,3cyclictest1975-21/usr/sbin/munin08:25:112
35299140,5cyclictest0-21swapper/008:35:270
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional