You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 19:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Sun Nov 16, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3196721230,1sleep031965-21ssh11:38:520
2083211896,16sleep10-21swapper/107:06:401
22812115104,6sleep20-21swapper/207:09:132
2012211594,15sleep30-21swapper/307:05:453
1994210480,19sleep00-21swapper/007:05:300
301402750,0sleep20-21swapper/211:02:352
131862750,1sleep121-21rcuc/111:52:351
209152720,0sleep20-21swapper/207:54:552
196362670,0sleep30-21swapper/312:33:473
155052660,1sleep121-21rcuc/110:13:501
68852560,1sleep011-21rcuc/009:30:170
237799222,13cyclictest0-21swapper/311:08:053
2377992219,2cyclictest30186-21kworker/u8:010:34:053
2377992219,2cyclictest24405-21kworker/u8:109:52:053
2377992218,3cyclictest19208-21kworker/u8:110:27:053
2376992219,2cyclictest20141-21kworker/u8:011:56:052
2376992217,3cyclictest22465-21chrt10:55:132
2377992118,2cyclictest8665-21kworker/u8:008:44:053
2377992118,2cyclictest5476-21kworker/u8:111:18:023
2377992118,2cyclictest5476-21kworker/u8:111:18:013
2377992118,2cyclictest3882-21kworker/u8:010:10:063
2377992118,2cyclictest3882-21kworker/u8:010:03:053
2377992118,2cyclictest27210-21kworker/u8:311:10:053
2377992118,2cyclictest27210-21kworker/u8:311:10:043
2377992118,2cyclictest27071-21kworker/u8:207:16:043
2377992118,2cyclictest20141-21kworker/u8:012:00:043
2377992118,2cyclictest20141-21kworker/u8:011:35:043
2377992118,2cyclictest20141-21kworker/u8:010:54:063
2377992118,2cyclictest19208-21kworker/u8:110:36:063
2377992118,2cyclictest13080-21kworker/u8:109:10:063
2377992117,3cyclictest31340-21kworker/u8:108:49:033
2377992117,3cyclictest11411-21kworker/u8:107:28:043
237799211,2cyclictest26688-21rm11:33:053
237699212,2cyclictest0-21swapper/211:45:052
237699210,19cyclictest0-21swapper/208:25:142
2375992118,2cyclictest20141-21kworker/u8:011:08:051
2377992017,2cyclictest8205-21kworker/u8:211:54:043
2377992017,2cyclictest27071-21kworker/u8:207:45:043
2377992017,2cyclictest21457-21kworker/u8:112:21:053
2377992017,2cyclictest21457-21kworker/u8:112:05:053
2377992017,2cyclictest21249-21kworker/u8:208:06:053
2377992017,2cyclictest20141-21kworker/u8:011:01:053
2377992017,2cyclictest19689-21kworker/u8:009:23:053
2377992017,2cyclictest19208-21kworker/u8:110:18:053
2377992017,2cyclictest18046-21kworker/u8:112:38:063
2377992017,2cyclictest17258-21kworker/u8:007:54:013
2377992017,2cyclictest14746-21kworker/u8:110:58:053
2377992017,2cyclictest13080-21kworker/u8:109:08:053
2377992017,2cyclictest13080-21kworker/u8:109:08:043
237799201,2cyclictest19387-21ssh09:44:053
2376992017,2cyclictest6639-21kworker/u8:212:29:052
2376992017,2cyclictest30186-21kworker/u8:010:35:052
2376992017,2cyclictest29902-21kworker/u8:210:05:052
2376992017,2cyclictest19689-21kworker/u8:009:27:052
2376992017,2cyclictest10753-21kworker/u8:208:50:022
237499200,2cyclictest0-21swapper/008:35:130
237499200,2cyclictest0-21swapper/007:55:130
70172190,1sleep10-21swapper/111:12:041
70172190,1sleep10-21swapper/111:12:031
2377991916,2cyclictest9257-21kworker/u8:010:43:053
2377991916,2cyclictest5476-21kworker/u8:111:21:053
2377991916,2cyclictest3882-21kworker/u8:010:06:053
2377991916,2cyclictest30811-21kworker/u8:311:47:053
2377991916,2cyclictest30811-21kworker/u8:311:41:053
2377991916,2cyclictest27362-21kworker/u8:008:29:053
2377991916,2cyclictest21457-21kworker/u8:112:26:043
2377991916,2cyclictest13974-21kworker/u8:210:24:043
2377991916,2cyclictest13080-21kworker/u8:109:00:053
2377991916,2cyclictest13080-21kworker/u8:108:57:053
2377991916,2cyclictest11411-21kworker/u8:107:30:043
2377991916,2cyclictest11411-21kworker/u8:107:23:033
237799190,18cyclictest0-21swapper/312:18:043
2376991916,2cyclictest4305-21kworker/u8:209:36:062
2376991916,2cyclictest3882-21kworker/u8:010:02:052
2376991916,2cyclictest30186-21kworker/u8:010:31:052
2376991916,2cyclictest20141-21kworker/u8:011:11:052
2376991916,2cyclictest20141-21kworker/u8:011:11:042
2376991916,2cyclictest16593-21kworker/u8:211:27:042
2376991916,2cyclictest10890-21kworker/u8:007:32:042
2376991914,3cyclictest29331-21ssh12:10:142
237699190,2cyclictest0-21swapper/209:44:052
237699190,17cyclictest0-21swapper/207:35:142
2375991916,2cyclictest21786-21kworker/u8:111:33:051
158622190,0sleep00-21swapper/011:20:570
2377991815,2cyclictest27362-21kworker/u8:008:31:053
2377991815,2cyclictest27071-21kworker/u8:207:41:043
2377991815,2cyclictest24405-21kworker/u8:109:55:043
2377991815,2cyclictest21249-21kworker/u8:208:00:053
2377991815,2cyclictest17654-21kworker/u8:009:49:053
2377991815,2cyclictest11411-21kworker/u8:107:10:043
2377991815,2cyclictest10890-21kworker/u8:007:35:043
237799180,2cyclictest0-21swapper/310:47:043
237799180,2cyclictest0-21swapper/309:32:043
2376991815,2cyclictest21457-21kworker/u8:112:08:052
2376991815,2cyclictest20141-21kworker/u8:011:38:062
2376991815,2cyclictest18046-21kworker/u8:112:37:052
2376991815,2cyclictest17654-21kworker/u8:009:45:052
237699180,5cyclictest0-21swapper/209:10:022
237699180,2cyclictest0-21swapper/211:33:052
237699180,14cyclictest0-21swapper/210:16:052
237599180,2cyclictest0-21swapper/108:25:131
2377991714,2cyclictest28207-21kworker/u8:212:11:053
2377991714,2cyclictest20141-21kworker/u8:011:58:053
2377991714,2cyclictest19689-21kworker/u8:009:25:053
2377991714,2cyclictest10753-21kworker/u8:208:52:053
237799170,2cyclictest0-21swapper/309:35:053
237799170,2cyclictest0-21swapper/308:14:043
2376991714,2cyclictest6639-21kworker/u8:212:31:052
2376991714,2cyclictest4910-21kworker/u8:208:43:052
2376991714,2cyclictest4910-21kworker/u8:208:38:042
2376991714,2cyclictest3882-21kworker/u8:010:12:052
2376991714,2cyclictest3377-21kworker/u8:210:48:042
2376991714,2cyclictest27362-21kworker/u8:008:34:052
2376991714,2cyclictest27362-21kworker/u8:008:13:042
2376991714,2cyclictest25776-21kworker/u8:209:22:042
2376991714,2cyclictest21249-21kworker/u8:208:03:052
2376991714,2cyclictest20141-21kworker/u8:011:53:042
2376991714,2cyclictest19689-21kworker/u8:009:15:042
2376991714,2cyclictest13080-21kworker/u8:108:58:052
2376991714,2cyclictest1211-21kworker/u8:012:20:052
2376991714,2cyclictest11411-21kworker/u8:107:27:052
237699170,2cyclictest0-21swapper/209:32:042
237699170,16cyclictest0-21swapper/210:50:052
2375991714,2cyclictest18046-21kworker/u8:112:39:051
2375991714,2cyclictest17654-21kworker/u8:009:50:021
2375991713,3cyclictest4305-21kworker/u8:209:32:051
237799160,2cyclictest0-21swapper/311:28:043
2376991613,2cyclictest31340-21kworker/u8:108:47:052
2376991613,2cyclictest21249-21kworker/u8:208:19:052
2376991613,2cyclictest21249-21kworker/u8:207:59:052
2376991613,2cyclictest13974-21kworker/u8:210:28:052
2376991613,2cyclictest13080-21kworker/u8:109:03:042
237699160,2cyclictest0-21swapper/212:17:042
237699160,2cyclictest0-21swapper/211:23:052
237699160,2cyclictest0-21swapper/211:19:052
237699160,2cyclictest0-21swapper/211:19:052
237699160,2cyclictest0-21swapper/210:22:042
237699160,11cyclictest0-21swapper/211:07:052
2375991613,2cyclictest30811-21kworker/u8:311:44:041
2375991613,2cyclictest21457-21kworker/u8:112:18:041
2375991613,2cyclictest20141-21kworker/u8:011:19:051
2375991613,2cyclictest20141-21kworker/u8:011:19:041
2375991613,2cyclictest13080-21kworker/u8:109:11:051
237599161,13cyclictest221ktimersoftd/110:45:121
237499160,2cyclictest0-21swapper/010:00:200
237799150,2cyclictest0-21swapper/309:18:053
237799150,2cyclictest0-21swapper/308:36:023
2376991512,2cyclictest27071-21kworker/u8:207:18:042
2376991512,2cyclictest24405-21kworker/u8:109:59:042
2376991512,2cyclictest21249-21kworker/u8:208:05:052
2376991512,2cyclictest14461-21kworker/u8:107:46:042
237699150,2cyclictest0-21swapper/210:40:062
2375991512,2cyclictest4305-21kworker/u8:209:48:041
237599150,13cyclictest22259-21munin-node1
237599150,13cyclictest2151-21ps09:25:231
2374991510,3cyclictest13502-21ssh11:53:060
2377991411,2cyclictest21249-21kworker/u8:208:16:053
2376991411,2cyclictest27071-21kworker/u8:207:13:042
237699140,12cyclictest0-21swapper/209:53:152
2375991411,2cyclictest5476-21kworker/u8:111:22:051
2375991411,2cyclictest4910-21kworker/u8:208:41:051
2375991411,2cyclictest31340-21kworker/u8:108:46:051
2375991411,2cyclictest27362-21kworker/u8:008:14:051
2375991411,2cyclictest19689-21kworker/u8:009:35:051
2375991411,2cyclictest17258-21kworker/u8:007:52:011
2375991411,2cyclictest0-21swapper/111:37:591
2375991411,2cyclictest0-21swapper/110:25:281
2375991411,2cyclictest0-21swapper/109:16:561
237599140,2cyclictest0-21swapper/110:50:291
237599140,12cyclictest25955-21latency08:05:201
237599140,12cyclictest18917-21diskmemload09:40:161
237599140,12cyclictest0-21swapper/111:26:551
237499142,7cyclictest0-21swapper/011:45:000
237499142,7cyclictest0-21swapper/009:30:010
237499142,6cyclictest0-21swapper/012:09:590
237499142,6cyclictest0-21swapper/009:45:000
2374991410,3cyclictest21475-21ssh12:00:370
237499140,13cyclictest0-21swapper/012:35:190
237499140,12cyclictest23606-21basename11:30:140
53292130,1sleep00-21swapper/011:45:070
4612130,2sleep10-21swapper/112:13:461
237799130,2cyclictest0-21swapper/308:21:053
2376991310,2cyclictest0-21swapper/212:00:312
237699130,2cyclictest0-21swapper/211:43:072
237699130,2cyclictest0-21swapper/208:23:052
237699130,2cyclictest0-21swapper/207:22:012
2375991310,2cyclictest8919-21ssh12:21:461
2375991310,2cyclictest21457-21kworker/u8:112:01:051
2375991310,2cyclictest14746-21kworker/u8:110:57:051
2375991310,2cyclictest14461-21kworker/u8:107:38:001
2375991310,2cyclictest0-21swapper/111:01:061
2375991310,2cyclictest0-21swapper/110:40:261
2375991310,2cyclictest0-21swapper/110:05:181
2375991310,2cyclictest0-21swapper/110:02:391
237599130,12cyclictest0-21swapper/109:58:271
237599130,12cyclictest0-21swapper/108:15:121
237599130,12cyclictest0-21swapper/107:25:151
237499132,6cyclictest0-21swapper/009:05:010
237499131,8cyclictest18917-21diskmemload11:04:030
237499131,10cyclictest12947-21ssh10:45:130
2374991310,2cyclictest0-21swapper/012:31:270
237499130,2cyclictest0-21swapper/012:17:390
237499130,11cyclictest18917-21diskmemload11:18:100
237499130,11cyclictest18917-21diskmemload11:18:090
237699129,2cyclictest0-21swapper/209:05:232
237699129,2cyclictest0-21swapper/209:05:232
237699120,2cyclictest0-21swapper/207:41:432
237599129,2cyclictest6639-21kworker/u8:212:25:051
237599129,2cyclictest4910-21kworker/u8:208:36:011
237599129,2cyclictest31340-21kworker/u8:108:24:051
237599129,2cyclictest27071-21kworker/u8:207:43:051
237599129,2cyclictest11411-21kworker/u8:107:22:011
237599129,2cyclictest10890-21kworker/u8:007:33:051
237599129,2cyclictest10753-21kworker/u8:208:50:061
237599129,2cyclictest0-21swapper/112:08:251
237599129,2cyclictest0-21swapper/110:30:211
237599129,2cyclictest0-21swapper/108:00:151
237599120,2cyclictest0-21swapper/111:45:561
237599120,2cyclictest0-21swapper/109:23:161
237599120,2cyclictest0-21swapper/108:55:201
237599120,11cyclictest0-21swapper/112:32:261
237599120,11cyclictest0-21swapper/110:37:121
237599120,11cyclictest0-21swapper/110:16:061
237599120,11cyclictest0-21swapper/107:45:151
237499129,2cyclictest0-21swapper/011:55:560
237499129,2cyclictest0-21swapper/011:26:490
237499129,2cyclictest0-21swapper/007:30:180
237499127,3cyclictest20169-21munin-node0
237499122,7cyclictest0-21swapper/008:51:010
237499121,3cyclictest32729-21sh09:24:120
237499120,2cyclictest0-21swapper/012:20:490
237499120,2cyclictest0-21swapper/010:25:130
237499120,11cyclictest0-21swapper/012:25:460
237499120,11cyclictest0-21swapper/012:10:260
237499120,10cyclictest18917-21diskmemload09:49:180
237499120,10cyclictest18074-21ps07:45:200
214852120,1sleep00-21swapper/010:54:030
196792120,1sleep00-21swapper/010:17:560
150432120,2sleep10-21swapper/111:55:141
237599118,2cyclictest27071-21kworker/u8:207:12:031
237599118,2cyclictest13080-21kworker/u8:109:02:041
237599118,2cyclictest0-21swapper/107:15:221
237599110,9cyclictest2807-21munin-node1
237599110,9cyclictest0-21swapper/107:55:281
237599110,2cyclictest0-21swapper/109:05:211
237599110,2cyclictest0-21swapper/109:05:211
237499117,2cyclictest0-21swapper/009:19:010
237499116,3cyclictest23033-21sh10:20:390
237499113,6cyclictest0-21swapper/007:40:270
237499111,7cyclictest18917-21diskmemload09:58:520
237499111,7cyclictest0-21swapper/008:34:020
237499111,7cyclictest0-21swapper/007:23:590
237499111,3cyclictest11901-21pzem_voltage09:35:220
237499110,6cyclictest0-21swapper/010:39:590
237499110,6cyclictest0-21swapper/008:13:310
237499110,6cyclictest0-21swapper/008:04:590
249382100,1sleep00-21swapper/009:50:020
237499107,2cyclictest0-21swapper/010:58:390
237499107,2cyclictest0-21swapper/010:06:260
237499105,3cyclictest5512-21ssh11:10:250
237499105,3cyclictest5512-21ssh11:10:250
237499105,3cyclictest4415-21cron07:15:000
237499105,3cyclictest12593-21sh10:45:000
237499102,7cyclictest0-21swapper/008:56:020
237499102,6cyclictest0-21swapper/007:38:230
237499101,7cyclictest18917-21diskmemload11:09:050
237499100,7cyclictest0-21swapper/008:19:590
237499100,6cyclictest18917-21diskmemload10:10:240
237499100,3cyclictest32606-21munin-node0
182452100,1sleep00-21swapper/009:05:290
182452100,1sleep00-21swapper/009:05:290
23779994,3cyclictest21566-21fschecks_count07:55:163
23749994,3cyclictest9371-21tune2fs07:25:140
23749994,3cyclictest5196-21tune2fs07:15:180
23749994,3cyclictest5129-21runrttasks08:05:500
23749994,3cyclictest32051-21wc10:30:270
23749994,3cyclictest1285-21cut08:25:150
23749994,3cyclictest10141-21sensors_temp08:45:250
23749991,3cyclictest278-21jbd2/sda1-808:40:060
23749983,3cyclictest19363-21cpuspeed_turbos07:50:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional