You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 17:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Sat Nov 29, 2025 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
285522124113,6sleep30-21swapper/307:07:283
28472211897,16sleep00-21swapper/007:06:230
285512116105,6sleep20-21swapper/207:07:272
2846329382,6sleep10-21swapper/107:06:161
133032760,0sleep00-21swapper/012:02:280
2878199260,8cyclictest0-21swapper/109:30:121
2878599223,2cyclictest0-21swapper/312:20:043
2878599223,2cyclictest0-21swapper/312:20:043
2878599220,17cyclictest0-21swapper/311:07:043
2878399224,2cyclictest0-21swapper/210:54:042
2878399223,2cyclictest0-21swapper/209:53:052
28783992219,2cyclictest4205-21kworker/u8:009:38:042
28783992219,2cyclictest23254-21kworker/u8:208:21:042
28783992219,2cyclictest20596-21kworker/u8:108:45:042
28783992219,2cyclictest11913-21kworker/u8:010:56:012
2878399221,4cyclictest27807-21kworker/u8:112:20:052
2878399221,4cyclictest27807-21kworker/u8:112:20:042
28781992217,3cyclictest332250irq/24-eth109:55:141
2878599212,2cyclictest0-21swapper/309:53:053
28785992118,2cyclictest30632-21kworker/u8:011:22:063
28785992118,2cyclictest30632-21kworker/u8:011:19:043
28785992118,2cyclictest23254-21kworker/u8:208:26:043
28785992118,2cyclictest22100-21kworker/u8:209:33:043
28785992118,2cyclictest22100-21kworker/u8:209:24:013
28785992118,2cyclictest21358-21kworker/u8:209:59:043
2878599211,2cyclictest0-21swapper/310:54:043
2878399212,4cyclictest0-21swapper/210:26:042
28783992118,2cyclictest8471-21kworker/u8:009:22:042
28783992118,2cyclictest8471-21kworker/u8:009:13:042
28783992118,2cyclictest8471-21kworker/u8:008:58:042
28783992118,2cyclictest772-21kworker/u8:207:36:052
28783992118,2cyclictest31725-21kworker/u8:010:05:042
28783992118,2cyclictest30632-21kworker/u8:011:22:012
28783992118,2cyclictest27807-21kworker/u8:112:26:042
28783992118,2cyclictest23254-21kworker/u8:208:25:052
28783992118,2cyclictest23254-21kworker/u8:208:16:042
28783992118,2cyclictest21508-21kworker/u8:111:45:052
28783992118,2cyclictest21358-21kworker/u8:210:00:042
28783992118,2cyclictest19009-21kworker/u8:007:26:012
28783992118,2cyclictest12854-21kworker/u8:212:16:042
28783992118,2cyclictest11941-21kworker/u8:207:48:052
28785992017,2cyclictest9483-21kworker/u8:310:30:013
28785992017,2cyclictest32285-21kworker/u8:210:46:023
28785992017,2cyclictest22567-21kworker/u8:107:20:053
28785992017,2cyclictest22567-21kworker/u8:107:20:043
28785992017,2cyclictest22567-21kworker/u8:107:19:053
28785992017,2cyclictest22567-21kworker/u8:107:19:043
28785992017,2cyclictest21508-21kworker/u8:111:46:053
28785992017,2cyclictest21475-21kworker/u8:110:28:053
28785992017,2cyclictest21358-21kworker/u8:210:03:053
28785992017,2cyclictest20596-21kworker/u8:108:49:053
28785992017,2cyclictest20596-21kworker/u8:108:33:043
28785992017,2cyclictest19009-21kworker/u8:007:14:043
28785992017,2cyclictest12934-21kworker/u8:212:39:043
28785992017,2cyclictest11913-21kworker/u8:010:56:063
28785992017,2cyclictest1091-21kworker/u8:011:52:053
28783992017,2cyclictest9483-21kworker/u8:310:33:042
28783992017,2cyclictest8471-21kworker/u8:009:00:052
28783992017,2cyclictest7481-21kworker/u8:012:32:052
28783992017,2cyclictest32285-21kworker/u8:210:47:052
28783992017,2cyclictest3146-21kworker/u8:211:38:022
28783992017,2cyclictest23254-21kworker/u8:208:36:022
28783992017,2cyclictest22100-21kworker/u8:209:41:042
28783992017,2cyclictest20596-21kworker/u8:108:42:042
28783992017,2cyclictest20596-21kworker/u8:108:31:042
2878399200,7cyclictest0-21swapper/212:10:022
28785991916,2cyclictest8471-21kworker/u8:009:19:043
28785991916,2cyclictest7946-21kworker/u8:111:58:043
28785991916,2cyclictest7481-21kworker/u8:012:31:053
28785991916,2cyclictest4744-21kworker/u8:210:15:043
28785991916,2cyclictest4744-21kworker/u8:210:15:043
28785991916,2cyclictest32285-21kworker/u8:211:04:013
28785991916,2cyclictest3146-21kworker/u8:211:38:053
28785991916,2cyclictest27807-21kworker/u8:112:28:043
28785991916,2cyclictest26325-21kworker/u8:110:11:043
28785991916,2cyclictest21475-21kworker/u8:110:37:043
28785991916,2cyclictest20596-21kworker/u8:108:10:053
28785991916,2cyclictest17040-21kworker/u8:008:03:043
28785991916,2cyclictest12854-21kworker/u8:212:18:043
28785991916,2cyclictest11941-21kworker/u8:207:59:043
2878599190,2cyclictest0-21swapper/310:23:053
28783991916,2cyclictest9483-21kworker/u8:310:16:062
28783991916,2cyclictest9483-21kworker/u8:310:16:052
28783991916,2cyclictest772-21kworker/u8:207:24:052
28783991916,2cyclictest772-21kworker/u8:207:24:042
28783991916,2cyclictest3146-21kworker/u8:211:31:042
28783991916,2cyclictest27334-21kworker/u8:010:35:052
28783991916,2cyclictest20596-21kworker/u8:108:06:042
28783991916,2cyclictest12934-21kworker/u8:212:38:052
28783991916,2cyclictest1091-21kworker/u8:012:01:052
2878399191,2cyclictest0-21swapper/210:23:052
280222190,1sleep128025-21cpuspeed_turbos10:00:141
28785991815,2cyclictest32285-21kworker/u8:211:10:053
28785991815,2cyclictest27334-21kworker/u8:010:40:053
28785991815,2cyclictest22100-21kworker/u8:209:43:043
28785991815,2cyclictest12719-21kworker/u8:109:49:043
28783991815,2cyclictest9074-21kworker/u8:107:44:042
28783991815,2cyclictest31725-21kworker/u8:010:10:042
28783991815,2cyclictest22567-21kworker/u8:107:11:042
28783991815,2cyclictest20596-21kworker/u8:108:50:052
28783991815,2cyclictest1091-21kworker/u8:011:52:012
28785991714,2cyclictest7946-21kworker/u8:112:00:043
28785991714,2cyclictest772-21kworker/u8:207:35:043
28785991714,2cyclictest4618-21kworker/u8:208:54:013
28785991714,2cyclictest29299-21kworker/u8:109:27:053
28785991714,2cyclictest23254-21kworker/u8:208:40:043
28785991714,2cyclictest19009-21kworker/u8:007:49:053
2878599170,2cyclictest0-21swapper/312:11:053
2878599170,2cyclictest0-21swapper/311:30:043
2878599170,2cyclictest0-21swapper/309:10:043
28783991714,2cyclictest8471-21kworker/u8:009:26:052
28783991714,2cyclictest32285-21kworker/u8:211:05:042
28783991714,2cyclictest12854-21kworker/u8:212:08:012
28783991714,2cyclictest11941-21kworker/u8:207:57:042
2878099171,3cyclictest6513-21sed12:30:150
28785991613,2cyclictest26325-21kworker/u8:110:06:043
2878599161,13cyclictest0-21swapper/307:35:003
28783991613,2cyclictest22567-21kworker/u8:107:16:062
28783991613,2cyclictest22567-21kworker/u8:107:16:052
28783991613,2cyclictest17040-21kworker/u8:008:04:022
2878199161,11cyclictest12837-21diskmemload10:35:261
2878099161,13cyclictest0-21swapper/011:20:170
28785991512,2cyclictest8471-21kworker/u8:009:04:053
28785991512,2cyclictest772-21kworker/u8:207:40:013
2878599150,7cyclictest0-21swapper/309:39:053
2878599150,2cyclictest0-21swapper/311:28:043
2878599150,2cyclictest0-21swapper/308:35:053
28783991512,2cyclictest772-21kworker/u8:207:33:042
28783991512,2cyclictest14549-21kworker/u8:107:54:052
2878399150,2cyclictest19678-21ssh09:16:022
2878399150,2cyclictest0-21swapper/211:28:042
2878199151,12cyclictest5230-21apt-get11:20:051
28781991510,4cyclictest0-21swapper/112:07:541
28781991510,3cyclictest5871-21perl10:45:171
28781991510,3cyclictest332250irq/24-eth110:53:001
2878199150,2cyclictest0-21swapper/111:15:141
2878199150,13cyclictest16497-21ssh09:47:291
2878199150,13cyclictest0-21swapper/111:37:551
2878199150,13cyclictest0-21swapper/110:06:001
28780991511,2cyclictest11-21rcuc/007:15:150
28780991511,2cyclictest11-21rcuc/007:15:140
2878099151,12cyclictest0-21swapper/010:29:120
28780991510,3cyclictest16502-21ssh09:47:300
28780991510,3cyclictest14867-21date11:30:000
2878099150,2cyclictest0-21swapper/010:32:050
2878099150,13cyclictest12837-21diskmemload11:06:250
2878099150,13cyclictest11-21rcuc/010:55:120
2878099150,13cyclictest0-21swapper/011:25:180
2878099150,13cyclictest0-21swapper/010:48:090
2878099150,13cyclictest0-21swapper/009:40:120
2878599141,7cyclictest0-21swapper/307:55:003
28785991411,2cyclictest17040-21kworker/u8:008:08:043
2878599140,2cyclictest0-21swapper/308:22:053
2878599140,2cyclictest0-21swapper/307:28:043
2878599140,13cyclictest0-21swapper/308:17:043
2878599140,12cyclictest18933-21ssh12:09:023
28783991411,2cyclictest0-21swapper/211:58:012
28783991411,2cyclictest0-21swapper/211:18:112
28783991410,2cyclictest0-21swapper/208:10:162
28781991411,2cyclictest0-21swapper/112:33:141
28781991411,2cyclictest0-21swapper/111:13:051
28781991411,2cyclictest0-21swapper/110:33:441
28781991411,2cyclictest0-21swapper/109:27:351
28781991411,2cyclictest0-21swapper/109:11:341
2878199140,3cyclictest0-21swapper/107:35:181
2878199140,2cyclictest12837-21diskmemload11:58:541
2878199140,2cyclictest12837-21diskmemload10:15:551
2878199140,2cyclictest12837-21diskmemload10:15:541
2878199140,2cyclictest0-21swapper/112:20:251
2878199140,2cyclictest0-21swapper/112:20:241
2878199140,13cyclictest0-21swapper/112:27:041
2878199140,12cyclictest6471-21chrt09:36:291
2878199140,12cyclictest18688-21ssh11:33:591
2878199140,12cyclictest12837-21diskmemload12:11:481
2878199140,12cyclictest12837-21diskmemload11:02:481
2878199140,12cyclictest12837-21diskmemload10:20:061
2878199140,12cyclictest0-21swapper/111:46:061
2878199140,12cyclictest0-21swapper/111:26:271
2878099149,3cyclictest9463-21ssh10:15:090
2878099149,3cyclictest9463-21ssh10:15:080
2878099149,3cyclictest22781-21sh11:03:300
2878099141,12cyclictest11-21rcuc/011:11:400
28780991411,2cyclictest0-21swapper/012:16:460
2878099141,11cyclictest11-21rcuc/010:13:360
2878099140,2cyclictest0-21swapper/009:20:150
2878099140,12cyclictest6570-21iostat11:55:170
2878099140,12cyclictest0-21swapper/012:05:490
2878099140,12cyclictest0-21swapper/011:52:070
2878099140,12cyclictest0-21swapper/008:10:170
284832140,3sleep00-21swapper/009:25:260
2878599130,2cyclictest0-21swapper/311:40:123
28783991310,2cyclictest0-21swapper/211:14:022
2878399130,2cyclictest0-21swapper/209:58:042
2878399130,12cyclictest0-21swapper/211:40:112
2878399130,12cyclictest0-21swapper/209:49:012
28781991310,2cyclictest12837-21diskmemload12:15:261
28781991310,2cyclictest12837-21diskmemload09:50:381
28781991310,2cyclictest0-21swapper/112:04:491
28781991310,2cyclictest0-21swapper/111:44:291
28781991310,2cyclictest0-21swapper/111:08:271
28781991310,2cyclictest0-21swapper/110:57:371
28781991310,2cyclictest0-21swapper/110:27:141
28781991310,2cyclictest0-21swapper/109:41:471
28781991310,2cyclictest0-21swapper/109:17:001
28781991310,2cyclictest0-21swapper/109:05:301
28781991310,2cyclictest0-21swapper/107:50:191
2878199130,2cyclictest12837-21diskmemload11:51:371
2878199130,2cyclictest0-21swapper/110:11:251
2878199130,2cyclictest0-21swapper/108:25:171
2878199130,12cyclictest0-21swapper/110:40:151
2878199130,11cyclictest12837-21diskmemload12:36:101
2878099131,11cyclictest12837-21diskmemload11:36:060
28780991310,2cyclictest9-21ksoftirqd/011:48:000
28780991310,2cyclictest12837-21diskmemload12:35:100
28780991310,2cyclictest12837-21diskmemload10:04:000
28780991310,2cyclictest0-21swapper/011:41:050
28780991310,2cyclictest0-21swapper/010:51:050
28780991310,2cyclictest0-21swapper/010:41:350
28780991310,2cyclictest0-21swapper/010:20:250
28780991310,2cyclictest0-21swapper/009:58:510
28780991310,2cyclictest0-21swapper/009:35:570
28780991310,2cyclictest0-21swapper/009:17:110
28780991310,2cyclictest0-21swapper/009:13:260
28780991310,2cyclictest0-21swapper/009:05:240
28780991310,2cyclictest0-21swapper/007:25:170
28780991310,2cyclictest0-21swapper/007:20:210
28780991310,2cyclictest0-21swapper/007:20:210
2878099130,2cyclictest0-21swapper/012:22:080
2878099130,2cyclictest0-21swapper/012:22:070
2878099130,2cyclictest0-21swapper/011:16:400
2878099130,2cyclictest0-21swapper/010:38:160
2878099130,12cyclictest18664-21copy08:03:400
2878099130,12cyclictest0-21swapper/012:13:000
2878099130,12cyclictest0-21swapper/009:54:390
2878599120,2cyclictest0-21swapper/308:57:043
2878399129,2cyclictest0-21swapper/209:31:532
2878399120,2cyclictest21408-21ssh11:01:132
2878199129,2cyclictest0-21swapper/107:20:161
2878199129,2cyclictest0-21swapper/107:20:161
2878199122,6cyclictest0-21swapper/108:25:001
2878199121,9cyclictest9427-21diskstats09:00:151
2878199120,2cyclictest0-21swapper/108:39:171
2878199120,2cyclictest0-21swapper/108:10:181
2878199120,11cyclictest0-21swapper/109:20:301
2878199120,10cyclictest0-21swapper/108:05:201
2878099129,2cyclictest12-21rcu_preempt12:29:580
2878099129,2cyclictest0-21swapper/009:31:310
2878099129,2cyclictest0-21swapper/009:00:270
2878099129,2cyclictest0-21swapper/008:35:180
2878099129,2cyclictest0-21swapper/007:30:110
2878099120,2cyclictest0-21swapper/010:05:320
2878099120,2cyclictest0-21swapper/007:45:180
2878099120,2cyclictest0-21swapper/007:35:230
2878099120,11cyclictest0-21swapper/008:05:180
2878599110,2cyclictest0-21swapper/309:06:043
2878199118,2cyclictest7394-21diskstats08:55:141
2878199118,2cyclictest0-21swapper/108:00:231
2878199118,2cyclictest0-21swapper/107:45:151
2878199111,8cyclictest31683-21iostat_ios07:15:171
2878199111,8cyclictest31683-21iostat_ios07:15:161
2878199111,8cyclictest28996-21apt-get07:10:121
2878199110,9cyclictest0-21swapper/108:45:151
2878199110,9cyclictest0-21swapper/107:40:241
2878199110,3cyclictest0-21swapper/108:40:141
2878199110,10cyclictest0-21swapper/108:50:221
2878099116,3cyclictest0-21swapper/008:50:180
2878099112,6cyclictest0-21swapper/008:42:590
2878099110,1cyclictest0-21swapper/008:30:200
2878399107,2cyclictest0-21swapper/210:41:162
2878399100,2cyclictest0-21swapper/209:07:042
2878199106,2cyclictest23154-21munin-node1
2878199102,6cyclictest0-21swapper/108:32:581
2878199101,7cyclictest15558-21sed07:55:151
2878199100,8cyclictest0-21swapper/107:25:131
2878099107,2cyclictest9-21ksoftirqd/007:55:250
2878099107,2cyclictest0-21swapper/007:40:250
2878099106,2cyclictest0-21swapper/008:25:190
2878099102,6cyclictest0-21swapper/008:48:010
287819994,3cyclictest5758-21wc07:30:181
287809994,3cyclictest8225-21timerandwakeup08:55:250
287809994,3cyclictest30957-21wc07:15:010
287809991,6cyclictest0-21swapper/007:50:250
287809990,2cyclictest0-21swapper/008:20:120
287809983,3cyclictest23381-21cpuspeed_turbos08:15:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional