You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-02-13 - 07:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot6s.osadl.org (updated Wed Feb 12, 2025 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25701211897,16sleep00-21swapper/007:08:300
255112116105,6sleep10-21swapper/107:06:031
257132114103,6sleep20-21swapper/207:08:402
2575929680,11sleep30-21swapper/307:09:133
271852730,1sleep30-21swapper/311:19:123
176522680,0sleep00-21swapper/010:33:070
97872560,0sleep30-21swapper/310:23:493
2584799360,34cyclictest0-21swapper/310:50:083
2584799360,34cyclictest0-21swapper/310:50:073
2584599361,33cyclictest0-21swapper/111:35:131
315252320,0sleep20-21swapper/208:50:002
19532260,1sleep00-21swapper/010:51:030
19532260,1sleep00-21swapper/010:51:020
208352230,1sleep20-21swapper/209:23:512
83962220,1sleep10-21swapper/109:10:181
2584499210,19cyclictest0-21swapper/007:50:170
25845992017,2cyclictest28169-21kworker/u8:008:10:041
25845992017,2cyclictest13858-21kworker/u8:111:43:041
25845992017,2cyclictest12780-21kworker/u8:010:30:031
25845992017,2cyclictest10021-21kworker/u8:209:49:051
2584599200,15cyclictest0-21swapper/109:38:021
25845991916,2cyclictest29938-21kworker/u8:112:21:051
25845991916,2cyclictest25525-21kworker/u8:209:01:041
25845991916,2cyclictest22845-21kworker/u8:010:05:041
25845991916,2cyclictest21617-21kworker/u8:108:47:041
2584599191,2cyclictest0-21swapper/108:40:121
25845991815,2cyclictest5008-21kworker/u8:111:04:051
25845991815,2cyclictest5008-21kworker/u8:111:04:041
25845991815,2cyclictest31717-21kworker/u8:008:56:041
25845991815,2cyclictest28784-21kworker/u8:011:29:031
25845991815,2cyclictest28169-21kworker/u8:008:30:051
25845991815,2cyclictest25525-21kworker/u8:208:37:031
25845991815,2cyclictest15818-21kworker/u8:109:59:041
25845991815,2cyclictest10021-21kworker/u8:210:10:041
2584599180,13cyclictest0-21swapper/112:12:051
2584799171,13cyclictest10661-21turbostat09:49:593
25845991714,2cyclictest9234-21kworker/u8:009:22:051
25845991714,2cyclictest5008-21kworker/u8:111:13:051
25845991714,2cyclictest29938-21kworker/u8:111:59:051
25845991714,2cyclictest28169-21kworker/u8:007:58:041
25845991714,2cyclictest28169-21kworker/u8:007:40:051
25845991714,2cyclictest2677-21kworker/u8:009:41:041
25845991714,2cyclictest24122-21kworker/u8:011:52:041
25845991714,2cyclictest23470-21kworker/u8:207:30:041
25845991714,2cyclictest11976-21kworker/u8:311:23:041
2584599170,16cyclictest0-21swapper/112:02:031
2584599170,15cyclictest0-21swapper/112:30:041
2584599170,12cyclictest0-21swapper/112:06:041
25844991712,3cyclictest0-21swapper/008:40:260
2584799161,3cyclictest29149-21ssh12:31:043
2584799161,13cyclictest20138-21ssh09:22:433
2584699162,8cyclictest28473-21ssh10:10:012
25845991613,2cyclictest3269-21kworker/u8:109:15:041
25845991613,2cyclictest26492-21kworker/u8:110:19:031
25845991613,2cyclictest25525-21kworker/u8:208:51:041
25845991613,2cyclictest23470-21kworker/u8:207:29:051
25845991613,2cyclictest18323-21kworker/u8:012:28:041
25845991613,2cyclictest1812-21kworker/u8:111:33:051
25845991613,2cyclictest15369-21kworker/u8:208:19:051
25845991613,2cyclictest12780-21kworker/u8:010:51:031
25845991613,2cyclictest12780-21kworker/u8:010:51:031
25845991613,2cyclictest1131-21kworker/u8:010:22:051
25844991611,3cyclictest20577-21sh10:00:150
2584499161,13cyclictest0-21swapper/012:20:150
2584799152,6cyclictest0-21swapper/311:10:583
2584799150,13cyclictest0-21swapper/311:08:243
2584699152,7cyclictest0-21swapper/207:50:012
2584699151,3cyclictest4923-21sed07:40:122
2584699151,12cyclictest27729-21munin-node2
25846991511,2cyclictest2066-21apt-get07:35:072
25846991510,3cyclictest10289-21ssh09:12:232
2584699150,14cyclictest0-21swapper/210:52:552
2584699150,14cyclictest0-21swapper/210:52:542
2584699150,13cyclictest0-21swapper/211:31:582
2584699150,13cyclictest0-21swapper/210:30:552
25845991512,2cyclictest28169-21kworker/u8:007:51:051
25845991512,2cyclictest28169-21kworker/u8:007:37:051
25845991512,2cyclictest23470-21kworker/u8:207:15:051
25845991512,2cyclictest12845-21kworker/u8:108:03:051
2584599150,2cyclictest0-21swapper/110:47:041
2584599150,2cyclictest0-21swapper/110:38:041
2584599150,13cyclictest0-21swapper/112:39:151
25844991512,2cyclictest7169-21diskmemload11:05:350
25844991510,3cyclictest0-21swapper/009:25:080
2584499150,2cyclictest0-21swapper/009:53:250
2584499150,2cyclictest0-21swapper/009:47:360
2584499150,14cyclictest0-21swapper/012:34:150
2584499150,13cyclictest0-21swapper/010:26:260
2584499150,13cyclictest0-21swapper/010:18:450
314352140,1sleep00-21swapper/010:12:160
2584799149,3cyclictest5951-21smartctl09:05:153
2584799149,3cyclictest26077-21aten2.4_r3power07:10:123
2584799142,7cyclictest21302-21copy08:25:013
2584799142,7cyclictest21302-21copy08:25:003
2584799142,6cyclictest0-21swapper/309:55:003
25847991411,2cyclictest0-21swapper/311:20:143
25847991411,2cyclictest0-21swapper/310:55:133
25847991411,2cyclictest0-21swapper/310:55:133
2584799140,13cyclictest0-21swapper/312:20:143
2584799140,13cyclictest0-21swapper/310:00:033
2584799140,13cyclictest0-21swapper/309:35:133
2584699149,3cyclictest0-21swapper/211:00:182
2584699149,3cyclictest0-21swapper/211:00:172
2584699149,2cyclictest0-21swapper/211:24:482
2584699142,7cyclictest21647-21copy09:25:002
25846991411,2cyclictest0-21swapper/210:44:122
25846991410,2cyclictest0-21swapper/212:05:112
2584699140,3cyclictest22855-21ssh12:24:352
2584699140,2cyclictest22537-21ssh10:38:052
2584699140,2cyclictest0-21swapper/211:55:152
2584699140,12cyclictest7169-21diskmemload11:47:082
2584699140,12cyclictest0-21swapper/212:30:132
2584699140,12cyclictest0-21swapper/212:10:532
2584699140,12cyclictest0-21swapper/212:03:212
2584699140,12cyclictest0-21swapper/211:12:082
2584699140,12cyclictest0-21swapper/209:47:552
25845991411,2cyclictest0-21swapper/111:16:561
2584599140,2cyclictest0-21swapper/111:46:051
2584599140,2cyclictest0-21swapper/110:58:061
2584599140,2cyclictest0-21swapper/110:58:051
2584599140,2cyclictest0-21swapper/107:11:041
2584599140,13cyclictest0-21swapper/110:26:061
2584599140,13cyclictest0-21swapper/110:02:361
2584599140,12cyclictest0-21swapper/109:33:071
25844991411,2cyclictest0-21swapper/012:18:550
25844991411,2cyclictest0-21swapper/011:48:160
25844991411,2cyclictest0-21swapper/011:18:450
25844991411,2cyclictest0-21swapper/009:59:160
25844991411,2cyclictest0-21swapper/007:45:150
2584499140,2cyclictest6049-21ssh11:31:050
2584499140,2cyclictest1221-21ssh12:00:460
2584499140,2cyclictest0-21swapper/011:50:190
2584499140,2cyclictest0-21swapper/011:38:260
2584499140,2cyclictest0-21swapper/009:17:250
2584499140,12cyclictest16226-21sh11:41:490
2584499140,12cyclictest0-21swapper/012:37:200
2584499140,12cyclictest0-21swapper/010:08:350
2584499140,12cyclictest0-21swapper/009:42:420
2584499140,12cyclictest0-21swapper/009:35:060
273552130,2sleep20-21swapper/209:31:102
268702130,2sleep10-21swapper/110:42:151
2584799138,3cyclictest11160-21ssh10:25:343
2584799132,6cyclictest0-21swapper/312:00:003
2584799132,6cyclictest0-21swapper/308:10:003
2584799132,6cyclictest0-21swapper/307:45:003
2584799131,3cyclictest0-21swapper/311:00:063
2584799131,3cyclictest0-21swapper/311:00:053
25847991310,2cyclictest0-21swapper/312:37:183
25847991310,2cyclictest0-21swapper/310:07:143
25847991310,2cyclictest0-21swapper/309:43:043
2584799130,2cyclictest0-21swapper/312:29:333
2584799130,2cyclictest0-21swapper/310:47:053
2584699139,2cyclictest16804-21ssh11:08:092
2584699138,3cyclictest0-21swapper/212:25:002
2584699132,7cyclictest0-21swapper/208:00:012
2584699132,6cyclictest0-21swapper/207:20:002
25846991310,2cyclictest0-21swapper/212:36:002
25846991310,2cyclictest0-21swapper/211:35:242
25846991310,2cyclictest0-21swapper/210:47:092
25846991310,2cyclictest0-21swapper/210:19:002
25846991310,2cyclictest0-21swapper/210:12:252
25846991310,2cyclictest0-21swapper/210:02:422
25846991310,2cyclictest0-21swapper/209:50:242
25846991310,2cyclictest0-21swapper/208:25:042
25846991310,2cyclictest0-21swapper/208:25:032
2584699130,2cyclictest18088-21ssh09:56:402
2584699130,2cyclictest0-21swapper/211:52:312
2584699130,2cyclictest0-21swapper/211:41:382
2584699130,2cyclictest0-21swapper/210:20:282
2584699130,2cyclictest0-21swapper/209:37:042
2584699130,2cyclictest0-21swapper/209:15:042
2584699130,12cyclictest0-21swapper/212:17:072
2584699130,12cyclictest0-21swapper/210:58:302
2584699130,12cyclictest0-21swapper/210:58:292
2584699130,12cyclictest0-21swapper/209:41:052
2584699130,11cyclictest0-21swapper/211:25:082
25845991310,2cyclictest7169-21diskmemload12:17:271
2584599130,2cyclictest0-21swapper/109:50:151
2584599130,2cyclictest0-21swapper/109:25:031
2584599130,2cyclictest0-21swapper/108:20:051
2584599130,2cyclictest0-21swapper/108:20:041
2584599130,2cyclictest0-21swapper/107:20:181
2584499134,2cyclictest0-21swapper/007:20:150
25844991310,2cyclictest19004-21irqstats11:10:130
2584499130,2cyclictest7169-21diskmemload11:00:190
2584499130,2cyclictest7169-21diskmemload11:00:190
2584499130,2cyclictest0-21swapper/011:58:050
2584499130,2cyclictest0-21swapper/009:11:560
2584499130,12cyclictest0-21swapper/011:28:250
2584499130,12cyclictest0-21swapper/010:20:260
2584499130,11cyclictest7169-21diskmemload09:21:400
63282120,2sleep37169-21diskmemload11:31:323
2584799129,2cyclictest0-21swapper/310:19:333
2584799129,2cyclictest0-21swapper/309:12:543
2584799122,7cyclictest0-21swapper/308:15:583
2584799122,6cyclictest0-21swapper/312:04:593
2584799120,11cyclictest0-21swapper/310:36:433
2584799120,11cyclictest0-21swapper/310:12:543
2584799120,11cyclictest0-21swapper/309:27:543
2584699129,2cyclictest0-21swapper/209:05:192
2584699128,2cyclictest25248-21ssh11:16:122
2584699122,7cyclictest0-21swapper/208:24:022
2584699122,7cyclictest0-21swapper/208:24:012
2584699120,2cyclictest0-21swapper/210:28:332
2584699120,2cyclictest0-21swapper/207:20:212
2584699120,11cyclictest0-21swapper/208:35:172
2584699120,11cyclictest0-21swapper/207:10:242
2584599129,2cyclictest28169-21kworker/u8:008:28:041
2584599129,2cyclictest28169-21kworker/u8:008:28:041
2584599129,2cyclictest0-21swapper/111:05:041
2584599120,11cyclictest0-21swapper/107:45:261
2584499129,2cyclictest29713-21ssh10:46:170
2584499129,2cyclictest0-21swapper/012:26:310
2584499129,2cyclictest0-21swapper/012:13:260
2584499129,2cyclictest0-21swapper/012:05:150
2584499129,2cyclictest0-21swapper/010:41:190
2584499129,2cyclictest0-21swapper/009:30:510
2584499129,2cyclictest0-21swapper/008:00:210
2584499129,2cyclictest0-21swapper/007:15:120
2584499122,6cyclictest0-21swapper/007:40:010
2584499121,9cyclictest15172-21http08:10:010
2584499121,6cyclictest0-21swapper/009:05:000
2584499120,2cyclictest7169-21diskmemload10:55:070
2584499120,2cyclictest7169-21diskmemload10:55:060
2584499120,2cyclictest7169-21diskmemload10:36:210
2584499120,11cyclictest0-21swapper/011:22:020
2584499120,11cyclictest0-21swapper/008:35:160
50522110,2sleep30-21swapper/312:05:473
2584799118,2cyclictest0-21swapper/311:39:133
2584799118,2cyclictest0-21swapper/309:55:353
2584799118,2cyclictest0-21swapper/309:00:143
2584799112,6cyclictest0-21swapper/311:50:193
2584799112,5cyclictest0-21swapper/309:15:253
2584799110,3cyclictest18630-21ssh10:34:373
2584799110,2cyclictest0-21swapper/312:17:533
2584799110,10cyclictest0-21swapper/311:43:443
2584799110,10cyclictest0-21swapper/309:30:103
2584699118,2cyclictest0-21swapper/208:10:122
2584599116,3cyclictest5373-21munin-node1
2584499116,3cyclictest30875-21cat08:45:280
2584499111,5cyclictest0-21swapper/008:55:000
2584499110,9cyclictest16151-21latency08:10:160
207802110,1sleep0101ktimersoftd/008:20:260
207802110,1sleep0101ktimersoftd/008:20:250
201892110,1sleep30-21swapper/311:47:423
2584799107,2cyclictest0-21swapper/311:25:033
2584799107,2cyclictest0-21swapper/310:41:283
2584799107,2cyclictest0-21swapper/308:55:153
2584799107,2cyclictest0-21swapper/308:40:123
2584799105,3cyclictest842-21sshd08:54:123
2584799101,6cyclictest0-21swapper/307:45:113
2584799100,8cyclictest15159-21cron08:14:563
2584799100,4cyclictest0-21swapper/307:25:013
2584699107,2cyclictest0-21swapper/207:50:252
2584699105,3cyclictest2060-21cron07:34:592
2584699104,4cyclictest3187-21date09:00:012
2584699102,6cyclictest0-21swapper/208:50:272
2584699102,6cyclictest0-21swapper/207:58:152
2584499107,2cyclictest0-21swapper/008:30:080
2584499102,5cyclictest0-21swapper/007:32:580
2584499101,5cyclictest0-21swapper/007:30:010
2584499100,2cyclictest0-21swapper/007:35:070
258479994,3cyclictest8529-21tr12:10:093
258479994,3cyclictest711-21grep07:30:103
258479994,3cyclictest32483-21sh07:30:003
258479994,3cyclictest30195-21if_err_eth008:45:183
258479994,3cyclictest28810-21latency07:15:163
258479994,3cyclictest2528-21cpu07:35:163
258479994,3cyclictest21561-21basename08:25:123
258479994,3cyclictest21561-21basename08:25:113
258479994,3cyclictest13706-21proc_pri08:00:193
258479994,3cyclictest0-21swapper/307:55:173
258479992,6cyclictest0-21swapper/308:35:233
258479992,6cyclictest0-21swapper/308:30:583
258469996,2cyclictest0-21swapper/209:00:152
258469994,3cyclictest3745-21runrttasks08:15:502
258469992,5cyclictest32484-21sh07:30:002
258469992,5cyclictest0-21swapper/208:30:272
258459991,6cyclictest3745-21runrttasks08:05:181
258449994,3cyclictest10797-21aten2.4_r3power07:55:110
258449992,5cyclictest0-21swapper/009:05:230
258449991,5cyclictest0-21swapper/008:20:010
258449990,7cyclictest23274-21cron08:30:010
258449990,7cyclictest23274-21cron08:30:000
258479985,2cyclictest0-21swapper/307:50:023
258469985,2cyclictest0-21swapper/208:06:282
258449984,2cyclictest1292-21cpu08:55:110
258449981,5cyclictest0-21swapper/007:10:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional