You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 16:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Mon Nov 24, 2025 12:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
143379912321207,18cyclictest30118-1kworker/1:0H07:55:131
143369912131183,23cyclictest1827-21kworker/0:107:25:100
142502656612,28sleep10-21swapper/107:24:461
138912463417,31sleep00-21swapper/007:21:010
1433699240217,18cyclictest26769-21df07:55:140
1433799180171,6cyclictest0-21swapper/107:25:101
1267121360,2sleep012670-21sh10:05:370
2416521220,2sleep024164-21ssh11:59:130
956821110,4sleep01433699cyclictest11:44:010
306702950,2sleep10-21swapper/110:25:131
123782850,8sleep11433799cyclictest12:20:171
254002820,8sleep01433699cyclictest12:33:550
168272790,4sleep11433799cyclictest11:17:351
168272790,4sleep11433799cyclictest11:17:341
144932690,2sleep014494-21sh11:49:090
227762490,7sleep11433799cyclictest07:45:161
83032460,5sleep11433799cyclictest10:35:191
250482460,4sleep122-21ksoftirqd/112:33:231
327042420,4sleep01433699cyclictest11:00:250
14337994213,15cyclictest22-21ksoftirqd/110:15:431
1433799418,11cyclictest0-21swapper/110:20:041
14337994118,8cyclictest22-21ksoftirqd/107:52:331
14337994115,7cyclictest22-21ksoftirqd/111:23:431
1433799411,19cyclictest32369-21fgrep12:40:271
14337994017,7cyclictest22-21ksoftirqd/111:29:051
14337994017,6cyclictest22-21ksoftirqd/109:33:271
14337994015,14cyclictest22-21ksoftirqd/111:48:321
1433699401,20cyclictest10017-21diskstats09:30:150
1433699401,19cyclictest28591-21ssh10:55:440
14337993924,5cyclictest22-21ksoftirqd/108:19:541
14337993918,6cyclictest22-21ksoftirqd/109:35:131
14337993918,6cyclictest22-21ksoftirqd/109:10:221
14337993917,8cyclictest22-21ksoftirqd/108:42:281
14337993917,7cyclictest22-21ksoftirqd/109:46:471
14337993917,7cyclictest22-21ksoftirqd/108:48:081
14337993917,6cyclictest22-21ksoftirqd/112:08:081
14337993917,6cyclictest22-21ksoftirqd/111:05:131
14337993916,7cyclictest22-21ksoftirqd/111:57:481
14337993916,6cyclictest22-21ksoftirqd/112:35:311
14337993916,14cyclictest22-21ksoftirqd/111:41:061
14337993915,6cyclictest22-21ksoftirqd/110:43:001
14337993913,15cyclictest22-21ksoftirqd/110:56:011
14337993913,10cyclictest22-21ksoftirqd/109:55:181
1433699392,21cyclictest25492-21ssh10:53:130
14337993819,8cyclictest22-21ksoftirqd/112:27:131
14337993817,6cyclictest22-21ksoftirqd/111:50:251
14337993817,6cyclictest22-21ksoftirqd/110:00:141
14337993817,10cyclictest3234-21diskstats10:30:171
14337993816,7cyclictest22-21ksoftirqd/112:02:461
14337993816,6cyclictest22-21ksoftirqd/111:13:031
14337993815,6cyclictest22-21ksoftirqd/109:17:381
14337993814,14cyclictest22-21ksoftirqd/109:54:471
14337993813,14cyclictest22-21ksoftirqd/112:50:181
66652371,32sleep06675-21ssh12:47:550
205512371,4sleep020701-21latency_hist09:05:010
1433799372,8cyclictest11833-21iostat_ios10:05:231
14337993717,6cyclictest22-21ksoftirqd/111:31:011
14337993716,6cyclictest22-21ksoftirqd/112:48:401
14337993716,6cyclictest22-21ksoftirqd/109:40:271
14337993715,7cyclictest22-21ksoftirqd/111:02:511
14337993715,7cyclictest22-21ksoftirqd/110:50:181
14337993715,7cyclictest22-21ksoftirqd/110:13:101
14337993715,6cyclictest22-21ksoftirqd/109:25:251
14337993713,7cyclictest22-21ksoftirqd/109:23:311
1433699372,18cyclictest7001-21iostat_ios10:00:220
217012361,4sleep021725-21ntp_states10:15:260
14337993621,5cyclictest22-21ksoftirqd/108:31:151
14337993616,6cyclictest22-21ksoftirqd/112:19:191
14337993614,7cyclictest22-21ksoftirqd/108:24:461
14337993614,6cyclictest22-21ksoftirqd/110:49:031
14337993613,8cyclictest22-21ksoftirqd/108:28:531
14337993613,6cyclictest22-21ksoftirqd/111:35:061
291442351,30sleep129153-21iostat_ios08:00:211
1433799347,15cyclictest30118-1kworker/1:0H12:14:121
14337993413,7cyclictest22-21ksoftirqd/109:07:221
1433799341,16cyclictest19463-21irqstats09:00:311
1433799341,16cyclictest19463-21irqstats09:00:301
1433699342,5cyclictest17894-21ssh09:38:190
14336993417,12cyclictest27889-21rm11:29:390
1433799332,9cyclictest0-21swapper/108:59:141
14336993323,6cyclictest5193-21ssh11:05:240
1433699332,14cyclictest32058-21memory12:40:230
1433699331,14cyclictest9635-21ssh10:36:120
1433699325,4cyclictest81rcu_preempt12:22:540
14336993218,10cyclictest22775-21aten_r4power_po07:45:150
1433699321,4cyclictest81rcu_preempt09:10:220
1433699321,4cyclictest81rcu_preempt07:52:330
30832311,4sleep03095-21sh09:22:480
1433699312,4cyclictest81rcu_preempt12:35:310
1433699311,5cyclictest81rcu_preempt12:08:080
1433699311,5cyclictest81rcu_preempt09:40:260
1433699311,4cyclictest81rcu_preempt11:50:250
1433699311,4cyclictest81rcu_preempt11:31:010
1433699311,4cyclictest81rcu_preempt10:41:570
1433699311,4cyclictest81rcu_preempt09:46:480
1433699311,4cyclictest81rcu_preempt08:48:080
1433699311,26cyclictest22437-21diskmemload10:31:320
186612301,5sleep00-21swapper/010:12:200
1433799306,12cyclictest0-21swapper/108:36:411
1433799302,9cyclictest30924-21hddtemp_smartct08:05:181
1433799302,9cyclictest1117-21users08:10:301
1433799301,8cyclictest19099-21/usr/sbin/munin07:35:111
1433699306,4cyclictest81rcu_preempt11:13:040
1433699303,8cyclictest5692-21memory09:25:250
1433699301,6cyclictest81rcu_preempt10:21:350
1433699301,5cyclictest81rcu_preempt12:02:470
1433699301,4cyclictest81rcu_preempt12:19:200
21712291,4sleep081rcu_preempt10:29:320
204362291,5sleep00-21swapper/007:37:500
160822291,4sleep016216-21/usr/sbin/munin08:50:260
1433799292,8cyclictest16317-21users08:50:271
14337992914,12cyclictest22-21ksoftirqd/107:30:021
14337992910,6cyclictest22-21ksoftirqd/107:40:121
1433699293,21cyclictest31586-21sort08:05:300
14336992918,6cyclictest19582-21rm11:20:230
1433699291,4cyclictest81rcu_preempt11:37:360
1433699291,4cyclictest81rcu_preempt11:17:170
1433699291,4cyclictest81rcu_preempt11:17:170
1433699291,4cyclictest81rcu_preempt10:47:130
1433699291,4cyclictest81rcu_preempt09:17:390
126932291,5sleep00-21swapper/008:41:560
71342281,5sleep00-21swapper/008:27:530
299302281,5sleep00-21swapper/008:01:320
1433699285,4cyclictest81rcu_preempt12:27:020
1433699284,3cyclictest81rcu_preempt08:24:460
1433699282,6cyclictest3274-21ntp_states12:10:270
1433699281,4cyclictest81rcu_preempt12:50:180
1433699280,4cyclictest81rcu_preempt09:59:400
31962271,5sleep00-21swapper/008:16:140
221702271,5sleep00-21swapper/007:41:050
184682271,5sleep00-21swapper/007:32:150
1433699272,3cyclictest81rcu_preempt09:50:370
14336992717,6cyclictest18065-21chrt08:55:310
93812261,5sleep00-21swapper/008:35:100
1433699262,6cyclictest8005-21perl08:30:240
14336992617,6cyclictest3-21ksoftirqd/009:00:130
1433699261,3cyclictest81rcu_preempt09:07:210
14336992514,7cyclictest581-21cat08:10:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional