You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-28 - 20:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Sat Jun 28, 2025 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7852211492,17sleep00-21swapper/007:07:120
80562111100,6sleep20-21swapper/207:09:492
7897210885,17sleep30-21swapper/307:07:463
6276210897,6sleep10-21swapper/107:05:031
185482810,0sleep30-21swapper/311:47:173
298942730,0sleep00-21swapper/012:31:280
153662690,1sleep015368-21rm10:39:000
224542550,4sleep2809999cyclictest12:25:002
208112300,0sleep00-21swapper/010:10:340
809799231,2cyclictest26037-21ssh09:44:130
809799231,2cyclictest0-21swapper/010:40:420
809799231,2cyclictest0-21swapper/010:30:150
809799230,3cyclictest0-21swapper/011:25:090
809799230,3cyclictest0-21swapper/011:10:220
809999223,2cyclictest0-21swapper/210:30:052
809999220,20cyclictest0-21swapper/207:25:132
809799220,3cyclictest8516-21ssh09:25:270
809799220,3cyclictest724-21kworker/u8:210:56:230
809799220,3cyclictest5129-21runrttasks12:36:060
809799220,3cyclictest4546-21cpuspeed_turbos09:55:140
809799220,3cyclictest29097-21ssh10:20:140
809799220,3cyclictest26881-21ssh10:17:190
809799220,3cyclictest24419-21diskmemload11:55:550
809799220,3cyclictest23831-21ssh10:46:540
809799220,3cyclictest23735-21ssh12:25:220
809799220,3cyclictest20449-21ssh12:21:590
809799220,3cyclictest18181-21ssh10:09:030
809799220,3cyclictest10251-21munin-node0
809799220,3cyclictest0-21swapper/012:05:230
809799220,3cyclictest0-21swapper/011:47:550
809799220,3cyclictest0-21swapper/011:35:300
809799220,3cyclictest0-21swapper/011:21:110
809799220,3cyclictest0-21swapper/011:01:500
809799220,3cyclictest0-21swapper/009:47:420
809799220,3cyclictest0-21swapper/009:24:580
809799220,3cyclictest0-21swapper/008:25:060
809799220,2cyclictest0-21swapper/011:15:300
809799220,2cyclictest0-21swapper/010:27:170
8099992118,2cyclictest6136-21kworker/u8:209:34:012
8099992118,2cyclictest19368-21kworker/u8:011:55:052
8099992118,2cyclictest15846-21kworker/u8:207:38:052
809799210,3cyclictest581-21sensors09:50:270
809799210,3cyclictest5129-21runrttasks07:10:220
809799210,3cyclictest30102-21ssh10:53:580
809799210,3cyclictest19291-21ssh09:36:260
809799210,3cyclictest12457-21kworker/u8:112:16:500
809799210,3cyclictest0-21swapper/008:37:260
809799210,3cyclictest0-21swapper/008:24:350
809799210,2cyclictest0-21swapper/012:10:220
809799210,2cyclictest0-21swapper/011:52:260
809799210,2cyclictest0-21swapper/009:13:380
809799210,2cyclictest0-21swapper/007:55:150
8099992017,2cyclictest6261-21kworker/u8:212:11:042
8099992017,2cyclictest5733-21kworker/u8:007:34:012
8099992017,2cyclictest29529-21kworker/u8:112:33:052
8099992017,2cyclictest26063-21kworker/u8:311:24:052
8099992017,2cyclictest24952-21kworker/u8:009:21:042
8099992017,2cyclictest24559-21kworker/u8:107:52:052
8099992017,2cyclictest1957-21kworker/u8:108:32:022
8099992017,2cyclictest19368-21kworker/u8:012:04:042
8099992017,2cyclictest15846-21kworker/u8:208:11:042
8099992017,2cyclictest15364-21kworker/u8:010:45:042
8099992017,2cyclictest13013-21kworker/u8:210:10:042
809999200,2cyclictest0-21swapper/210:20:142
8097992010,8cyclictest0-21swapper/009:32:570
809799200,3cyclictest20355-21apt-get09:00:070
809799200,3cyclictest20355-21apt-get09:00:060
809799200,3cyclictest18558-21munin-node0
809799200,3cyclictest12630-21ssh11:09:030
809799200,3cyclictest0-21swapper/009:15:190
809799200,2cyclictest22632-21apt-get09:05:150
809799200,2cyclictest0-21swapper/012:01:300
809799200,2cyclictest0-21swapper/011:43:420
809799200,2cyclictest0-21swapper/011:34:270
809799200,2cyclictest0-21swapper/007:50:190
809799200,2cyclictest0-21swapper/007:20:140
8099991916,2cyclictest944-21kworker/u8:011:35:042
8099991916,2cyclictest724-21kworker/u8:211:17:052
8099991916,2cyclictest5733-21kworker/u8:007:42:052
8099991916,2cyclictest4369-21kworker/u8:010:02:052
8099991916,2cyclictest4369-21kworker/u8:009:56:052
8099991916,2cyclictest30336-21kworker/u8:008:07:052
8099991916,2cyclictest27049-21kworker/u8:110:42:022
8099991916,2cyclictest19368-21kworker/u8:012:36:012
8099991916,2cyclictest13013-21kworker/u8:210:07:042
809799192,12cyclictest0-21swapper/008:10:000
809799191,16cyclictest20919-21cut07:40:160
809799190,2cyclictest11059-21latency07:15:190
809799190,2cyclictest0-21swapper/008:15:230
8100991815,2cyclictest724-21kworker/u8:210:59:053
8100991815,2cyclictest1957-21kworker/u8:109:14:053
8099991815,2cyclictest8496-21kworker/u8:208:49:042
8099991815,2cyclictest724-21kworker/u8:211:04:042
8099991815,2cyclictest5733-21kworker/u8:007:17:052
8099991815,2cyclictest30336-21kworker/u8:008:22:042
8099991815,2cyclictest30336-21kworker/u8:008:16:052
8099991815,2cyclictest1957-21kworker/u8:109:06:042
8099991815,2cyclictest19368-21kworker/u8:012:25:042
8099991815,2cyclictest16684-21kworker/u8:009:00:022
8099991815,2cyclictest16684-21kworker/u8:009:00:012
8099991815,2cyclictest12515-21kworker/u8:107:22:042
8098991815,2cyclictest12457-21kworker/u8:112:15:041
809799180,2cyclictest22481-21apt-get07:45:140
8100991714,2cyclictest4369-21kworker/u8:009:56:013
8099991714,2cyclictest944-21kworker/u8:011:31:052
8099991714,2cyclictest32497-21kworker/u8:112:06:052
8099991714,2cyclictest30336-21kworker/u8:008:04:022
8099991714,2cyclictest27049-21kworker/u8:110:27:052
8099991714,2cyclictest24952-21kworker/u8:009:17:042
8099991714,2cyclictest16684-21kworker/u8:008:55:052
8099991714,2cyclictest15846-21kworker/u8:207:56:042
809999170,2cyclictest0-21swapper/210:15:052
809899170,12cyclictest0-21swapper/110:59:051
8100991612,2cyclictest31041-21kworker/u8:109:50:053
8099991613,2cyclictest9909-21kworker/u8:211:42:052
8099991613,2cyclictest27049-21kworker/u8:110:36:052
8099991613,2cyclictest1957-21kworker/u8:109:25:052
809899161,13cyclictest10323-21apt-get07:15:141
809799163,11cyclictest16392-21munin-node0
809799161,4cyclictest5129-21runrttasks07:35:140
809799161,4cyclictest16179-21on_ac_power07:30:000
8100991512,2cyclictest26425-21kworker/u8:111:02:053
810099150,13cyclictest20231-21pzem_current10:10:203
8099991512,2cyclictest6136-21kworker/u8:209:39:052
8099991512,2cyclictest1957-21kworker/u8:108:27:052
8099991512,2cyclictest0-21swapper/209:53:432
8099991510,3cyclictest0-21swapper/208:35:172
809899150,3cyclictest4715-21diskstats09:55:151
809799152,3cyclictest30898-21cpuspeed_turbos08:05:160
809799151,4cyclictest17270-21grep08:50:180
809799151,3cyclictest5129-21runrttasks08:30:150
809799151,3cyclictest15491-21wc08:45:210
809799151,3cyclictest12770-21tr08:40:160
809799151,3cyclictest0-21swapper/008:00:130
810099147,6cyclictest23994-21ssh10:15:053
810099141,12cyclictest32211-21ssh12:35:013
8100991411,2cyclictest27049-21kworker/u8:110:37:053
8100991411,2cyclictest0-21swapper/310:43:503
810099140,3cyclictest31326-21ssh09:49:483
810099140,3cyclictest28486-21ssh11:57:283
810099140,2cyclictest0-21swapper/310:53:593
810099140,13cyclictest0-21swapper/311:29:013
810099140,12cyclictest8874-21ssh09:25:313
810099140,12cyclictest0-21swapper/312:17:033
810099140,12cyclictest0-21swapper/311:30:003
810099140,12cyclictest0-21swapper/309:34:583
810099140,12cyclictest0-21swapper/309:18:193
8099991411,2cyclictest6136-21kworker/u8:209:45:042
8099991411,2cyclictest5733-21kworker/u8:007:11:042
8099991411,2cyclictest15846-21kworker/u8:207:48:042
809999140,2cyclictest0-21swapper/212:15:332
809999140,13cyclictest24419-21diskmemload11:08:332
809999140,13cyclictest0-21swapper/211:53:432
809999140,13cyclictest0-21swapper/211:46:222
8098991411,2cyclictest944-21kworker/u8:011:34:061
8098991411,2cyclictest26425-21kworker/u8:111:01:041
8098991411,2cyclictest0-21swapper/112:07:221
8098991411,2cyclictest0-21swapper/111:57:321
8098991411,2cyclictest0-21swapper/111:25:161
8098991411,2cyclictest0-21swapper/111:15:181
8098991411,2cyclictest0-21swapper/109:37:111
809899140,3cyclictest0-21swapper/107:35:241
809899140,13cyclictest0-21swapper/111:45:011
809899140,12cyclictest11788-21runrttasks09:29:571
809899140,12cyclictest0-21swapper/110:33:031
809899140,12cyclictest0-21swapper/109:30:321
181442140,3sleep318157-21cron11:14:593
8100991310,2cyclictest19368-21kworker/u8:012:38:053
8100991310,2cyclictest0-21swapper/312:27:183
8100991310,2cyclictest0-21swapper/311:41:293
8100991310,2cyclictest0-21swapper/310:31:093
8100991310,2cyclictest0-21swapper/310:20:593
8100991310,2cyclictest0-21swapper/310:07:183
8100991310,2cyclictest0-21swapper/310:03:013
8100991310,2cyclictest0-21swapper/308:15:013
8100991310,2cyclictest0-21swapper/307:15:183
810099130,2cyclictest0-21swapper/312:10:503
810099130,2cyclictest0-21swapper/311:50:283
810099130,2cyclictest0-21swapper/311:06:583
810099130,2cyclictest0-21swapper/308:00:193
810099130,12cyclictest0-21swapper/312:03:293
810099130,12cyclictest0-21swapper/309:37:013
8099991310,2cyclictest0-21swapper/209:41:162
809999130,2cyclictest24419-21diskmemload11:25:212
809999130,2cyclictest0-21swapper/210:56:222
809999130,12cyclictest0-21swapper/210:50:532
809899131,10cyclictest0-21swapper/110:45:461
8098991310,2cyclictest1957-21kworker/u8:108:25:041
8098991310,2cyclictest0-21swapper/112:10:141
8098991310,2cyclictest0-21swapper/110:54:131
8098991310,2cyclictest0-21swapper/110:10:121
8098991310,2cyclictest0-21swapper/109:12:551
809899130,2cyclictest24419-21diskmemload09:20:131
809899130,2cyclictest0-21swapper/110:25:331
809899130,12cyclictest0-21swapper/112:33:221
809899130,12cyclictest0-21swapper/112:25:181
809899130,12cyclictest0-21swapper/112:20:161
809899130,12cyclictest0-21swapper/112:03:111
809899130,12cyclictest0-21swapper/111:20:111
809899130,12cyclictest0-21swapper/111:10:261
809899130,12cyclictest0-21swapper/110:42:221
809899130,12cyclictest0-21swapper/110:37:421
809899130,11cyclictest24419-21diskmemload12:35:251
809899130,11cyclictest24419-21diskmemload11:35:271
809899130,11cyclictest24419-21diskmemload10:01:041
809899130,11cyclictest0-21swapper/110:05:241
809899130,11cyclictest0-21swapper/109:51:551
809899130,11cyclictest0-21swapper/108:30:241
809899130,11cyclictest0-21swapper/107:45:241
257002130,2sleep20-21swapper/209:10:212
188532130,1sleep318856-21cpuspeed_turbos08:55:143
810099129,2cyclictest19368-21kworker/u8:012:05:053
810099129,2cyclictest0-21swapper/310:45:083
810099129,2cyclictest0-21swapper/309:42:583
810099120,11cyclictest0-21swapper/312:20:363
810099120,11cyclictest0-21swapper/307:35:163
810099120,10cyclictest26063-21kworker/u8:311:22:013
809999124,5cyclictest0-21swapper/208:40:592
809999120,2cyclictest0-21swapper/211:11:062
809899129,2cyclictest0-21swapper/107:50:251
809899129,2cyclictest0-21swapper/107:34:391
809899121,9cyclictest12353-21munin-node1
809899121,3cyclictest4298-21munin-node1
809899121,3cyclictest0-21swapper/108:40:151
809899120,2cyclictest0-21swapper/111:51:351
809899120,2cyclictest0-21swapper/111:45:221
809899120,2cyclictest0-21swapper/110:15:261
809899120,2cyclictest0-21swapper/108:13:391
809899120,11cyclictest24419-21diskmemload09:46:031
809899120,11cyclictest0-21swapper/111:05:271
809899120,11cyclictest0-21swapper/110:22:121
809899120,11cyclictest0-21swapper/109:40:021
809899120,11cyclictest0-21swapper/109:15:041
809899120,11cyclictest0-21swapper/108:45:221
38952120,2sleep30-21swapper/310:26:463
810099118,2cyclictest0-21swapper/308:34:393
810099110,2cyclictest0-21swapper/311:19:433
810099110,1cyclictest0-21swapper/307:50:233
810099110,10cyclictest0-21swapper/307:25:173
809999117,2cyclictest17233-21irqstats08:50:192
809899118,2cyclictest0-21swapper/109:00:261
809899118,2cyclictest0-21swapper/109:00:251
809899111,8cyclictest0-21swapper/108:05:191
809899111,3cyclictest8317-21munin-node1
809899111,3cyclictest22722-21aten2.4-expect09:05:151
809899111,3cyclictest20468-21munin-node1
809899110,9cyclictest0-21swapper/108:00:141
810099107,2cyclictest0-21swapper/308:27:203
810099107,2cyclictest0-21swapper/308:21:403
810099100,9cyclictest15846-21kworker/u8:208:12:043
810099100,8cyclictest0-21swapper/311:35:533
810099100,2cyclictest0-21swapper/309:20:543
809899107,2cyclictest0-21swapper/108:50:001
809899101,8cyclictest0-21swapper/108:55:141
809899100,8cyclictest0-21swapper/107:55:241
809899100,8cyclictest0-21swapper/107:25:161
809899100,3cyclictest0-21swapper/108:15:281
809899100,2cyclictest10517-21apt-get08:35:161
81009995,3cyclictest30842-21aten2.4-expect08:05:153
81009990,8cyclictest0-21swapper/309:02:493
81009990,8cyclictest0-21swapper/309:02:483
81009990,8cyclictest0-21swapper/307:20:083
81009990,7cyclictest14679-21munin-node3
81009990,2cyclictest0-21swapper/308:53:413
81009985,2cyclictest0-21swapper/307:48:593
81009983,3cyclictest26743-21cpuspeed_turbos07:55:143
81009980,7cyclictest5733-21kworker/u8:007:44:053
81009980,7cyclictest0-21swapper/307:31:593
81009980,7cyclictest0-21swapper/307:10:273
81009980,2cyclictest0-21swapper/308:35:583
81009974,2cyclictest381ktimersoftd/309:05:143
81009970,6cyclictest0-21swapper/308:40:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional