You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-15 - 03:16
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot1.osadl.org (updated Tue Jul 15, 2025 00:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
811021810,0sleep32654199cyclictest23:10:123
265339916527,79cyclictest3730-21watchprog-ntop20:46:000
1132421640,1sleep02653399cyclictest00:28:380
265499915312,62cyclictest0-21swapper/723:18:297
2498321530,3sleep22653699cyclictest19:12:132
265339914711,53cyclictest10232-21chk_conn_tcp21:54:030
1797321470,0sleep30-21swapper/323:35:363
564021410,0sleep10-21swapper/119:59:561
265369914123,56cyclictest0-21swapper/219:38:082
265369913418,34cyclictest3730-21watchprog-ntop20:33:102
1596721340,1sleep415960-21chk_conn_tcp23:34:574
265369912727,55cyclictest0-21swapper/220:05:232
265339912687,16cyclictest0-21swapper/021:04:420
265339912624,65cyclictest188850irq/25-eth019:08:340
26536991256,33cyclictest0-21swapper/220:13:562
2449921230,1sleep6691ktimersoftd/623:04:546
265339912151,58cyclictest188850irq/25-eth022:26:070
2443121210,0sleep00-21swapper/019:11:550
1540121200,0sleep10-21swapper/121:22:251
584921190,1sleep114342-21chk_conn_tcp20:28:331
265339911866,19cyclictest3730-21watchprog-ntop22:36:160
1236021180,0sleep00-21swapper/021:43:400
265369911743,5cyclictest3730-21watchprog-ntop23:37:432
652721160,1sleep53729-21runrttasks00:00:385
265369911531,21cyclictest0-21swapper/221:22:522
265369911518,58cyclictest27157-21openvpn19:06:262
265369911329,21cyclictest0-21swapper/222:46:252
265339911323,21cyclictest27157-21openvpn19:16:350
2936621110,0sleep40-21swapper/421:27:014
265459911014,19cyclictest0-21swapper/520:31:025
265459911013,65cyclictest3730-21watchprog-ntop21:45:525
265459911013,65cyclictest3730-21watchprog-ntop21:45:525
265369911092,17cyclictest34-21ksoftirqd/221:37:192
26536991104,33cyclictest0-21swapper/223:52:402
265369910921,4cyclictest0-21swapper/220:45:282
265339910919,89cyclictest188850irq/25-eth022:16:300
26545991071,46cyclictest3730-21watchprog-ntop19:05:545
265339910720,43cyclictest2729-21QueuingRrdStrat21:46:560
265339910720,43cyclictest2729-21QueuingRrdStrat21:46:560
26533991071,22cyclictest0-21swapper/022:31:280
265459910625,4cyclictest0-21swapper/522:48:335
26549991058,48cyclictest0-21swapper/723:08:527
265499910449,17cyclictest0-21swapper/700:22:047
265499910213,64cyclictest0-21swapper/722:53:227
265369910233,42cyclictest0-21swapper/220:24:052
265339910225,16cyclictest0-21swapper/019:47:450
2754821010,1sleep627547-21chk_conn_tcp19:41:186
26549991019,78cyclictest0-21swapper/721:59:567
26545991012,54cyclictest5787-21chk_conn_tcp21:57:485
1679721010,1sleep216799-21timerwakeupswit23:45:052
265499910041,17cyclictest0-21swapper/722:23:597
265499910020,8cyclictest0-21swapper/723:24:217
265459910017,56cyclictest5841-21ssh00:26:525
265339910035,2cyclictest16165-21chk_conn_tcp23:40:550
1060621000,1sleep523508-21Collectd-Thread23:43:315
294732990,0sleep20-21swapper/222:22:282
26536999914,17cyclictest0-21swapper/219:28:552
26549999735,21cyclictest214550irq/30-eth2-TxR19:54:107
26536999756,19cyclictest34-21ksoftirqd/222:58:102
26536999729,44cyclictest3730-21watchprog-ntop20:00:032
26536999716,20cyclictest0-21swapper/223:19:012
23972970,0sleep60-21swapper/600:07:526
244572960,1sleep424453-21chk_conn_tcp21:47:284
244572960,1sleep424453-21chk_conn_tcp21:47:284
26545999428,23cyclictest0-21swapper/523:30:465
26536999453,16cyclictest34-21ksoftirqd/200:11:552
39402930,1sleep25787-21chk_conn_tcp20:41:322
2653399936,78cyclictest188850irq/25-eth023:46:160
317052920,24sleep0188850irq/25-eth022:12:080
26545999231,44cyclictest0-21swapper/519:12:185
2653399929,20cyclictest3730-21watchprog-ntop20:08:040
2653699917,81cyclictest3730-21watchprog-ntop19:02:092
26536999136,41cyclictest3730-21watchprog-ntop21:59:242
26549999035,1cyclictest3730-21watchprog-ntop22:28:157
26549999011,39cyclictest0-21swapper/700:10:187
2654599900,33cyclictest0-21swapper/521:25:325
26536998918,37cyclictest0-21swapper/222:52:502
26533998912,43cyclictest0-21swapper/021:02:340
26545998830,38cyclictest0-21swapper/521:53:315
26536998852,12cyclictest26462-21ssh22:43:452
26536998836,1cyclictest34-21ksoftirqd/222:33:042
26533998810,22cyclictest0-21swapper/019:57:220
26549998722,28cyclictest0-21swapper/722:09:017
2654999870,20cyclictest3730-21watchprog-ntop19:01:057
2654599877,15cyclictest0-21swapper/522:20:465
2653399870,26cyclictest0-21swapper/000:17:470
26545998625,38cyclictest23200-21Poller-Thread-623:37:115
26533998630,39cyclictest0-21swapper/020:01:070
163052860,1sleep416302-21chk_conn_tcp23:54:524
2654999853,39cyclictest0-21swapper/719:58:277
2654999851,61cyclictest0-21swapper/723:54:177
2654999850,60cyclictest0-21swapper/721:08:267
2654599853,49cyclictest0-21swapper/520:41:435
26545998525,40cyclictest0-21swapper/522:19:105
26545998522,1cyclictest5787-21chk_conn_tcp23:07:485
121932840,0sleep450-21rcuc/421:43:364
2654999820,18cyclictest0-21swapper/720:36:557
26533998222,1cyclictest0-21swapper/020:58:490
266182810,1sleep2331ktimersoftd/221:48:222
266182810,1sleep2331ktimersoftd/221:48:222
26549998114,27cyclictest0-21swapper/721:57:167
26545998121,19cyclictest0-21swapper/520:28:545
2654499810,33cyclictest0-21swapper/420:19:494
26536998126,18cyclictest8746-21chk_conn_tcp21:42:082
2653399811,37cyclictest0-21swapper/000:20:280
26545998022,22cyclictest0-21swapper/500:17:155
26545998012,22cyclictest0-21swapper/520:22:295
26536998038,1cyclictest3730-21watchprog-ntop20:58:172
87302790,0sleep30-21swapper/320:43:573
26544997939,1cyclictest9255-21chk_conn_tcp23:32:544
26544997934,44cyclictest0-21swapper/423:26:304
2654499790,34cyclictest0-21swapper/420:29:584
26536997913,60cyclictest0-21swapper/200:23:082
26533997935,0cyclictest16165-21chk_conn_tcp19:50:260
39822780,0sleep50-21swapper/523:51:175
284822780,0sleep40-21swapper/421:59:544
26549997820,41cyclictest0-21swapper/700:05:307
26536997836,22cyclictest0-21swapper/222:06:212
26533997810,39cyclictest7057-21chk_conn_tcp19:32:400
91972770,0sleep60-21swapper/620:44:166
295772770,0sleep60-21swapper/618:59:596
2654999770,71cyclictest14995-21chk_conn_tcp19:36:007
26545997736,16cyclictest0-21swapper/523:58:335
26545997720,19cyclictest0-21swapper/521:42:405
26536997732,2cyclictest0-21swapper/200:06:022
26533997731,33cyclictest0-21swapper/022:51:460
26533997721,55cyclictest0-21swapper/022:03:400
26545997617,39cyclictest3878-21chk_conn_tcp21:07:225
26536997631,6cyclictest3730-21watchprog-ntop19:32:082
26533997629,46cyclictest0-21swapper/022:56:020
26549997517,56cyclictest20843-21chk_conn_tcp00:04:267
26545997511,13cyclictest0-21swapper/519:30:005
26533997523,42cyclictest0-21swapper/022:04:450
26533997521,19cyclictest16495-21chk_conn_tcp19:42:570
26545997417,0cyclictest3730-21watchprog-ntop22:00:285
2654599740,14cyclictest0-21swapper/522:51:145
124792740,1sleep30-21swapper/323:34:163
26549997334,19cyclictest0-21swapper/719:41:217
26549997317,22cyclictest781ktimersoftd/721:19:397
26536997329,0cyclictest0-21swapper/221:14:192
26533997338,0cyclictest0-21swapper/019:27:510
2654999728,20cyclictest3730-21watchprog-ntop19:04:507
2654999720,34cyclictest0-21swapper/720:25:427
26545997234,3cyclictest0-21swapper/521:38:555
26545997227,34cyclictest0-21swapper/519:26:475
2654599720,40cyclictest12882-21ssh23:12:045
2654599720,31cyclictest16495-21chk_conn_tcp23:03:315
26536997229,4cyclictest0-21swapper/220:34:472
26533997223,42cyclictest0-21swapper/020:11:480
210372720,1sleep63729-21runrttasks00:22:196
26545997124,46cyclictest0-21swapper/522:29:195
26549997023,21cyclictest0-21swapper/723:02:277
2654999700,20cyclictest0-21swapper/720:50:167
2654599708,56cyclictest0-21swapper/521:13:475
26536997027,36cyclictest331ktimersoftd/223:11:002
24462700,1sleep450-21rcuc/421:50:574
2654999698,56cyclictest0-21swapper/720:16:377
26544996911,56cyclictest23267-21chk_conn_tcp19:54:104
26536996916,17cyclictest3730-21watchprog-ntop23:44:072
26536996911,22cyclictest0-21swapper/223:06:112
26536996910,16cyclictest3730-21watchprog-ntop19:52:022
2653399697,52cyclictest5787-21chk_conn_tcp23:16:200
26533996934,20cyclictest0-21swapper/019:22:280
26533996916,0cyclictest0-21swapper/023:22:130
2653399691,30cyclictest5787-21chk_conn_tcp21:16:590
258782690,3sleep32654199cyclictest23:48:103
26549996820,25cyclictest14342-21chk_conn_tcp23:30:147
26549996813,22cyclictest3730-21watchprog-ntop22:34:407
26544996822,45cyclictest0-21swapper/422:58:434
26536996847,3cyclictest0-21swapper/222:02:042
26536996836,10cyclictest11-21rcu_preempt19:42:252
2653399689,20cyclictest0-21swapper/021:11:390
181532680,0sleep60-21swapper/621:12:206
87812670,0sleep40-21swapper/419:05:024
2654599676,36cyclictest3590-21postmaster19:48:175
2654599672,4cyclictest0-21swapper/519:56:185
2654599670,34cyclictest0-21swapper/521:34:075
26533996720,20cyclictest0-21swapper/023:58:010
26549996624,30cyclictest0-21swapper/723:10:287
26544996629,1cyclictest3730-21watchprog-ntop21:54:354
2653699667,36cyclictest0-21swapper/222:12:132
112382660,0sleep30-21swapper/322:49:453
35142650,0sleep60-21swapper/620:13:236
26549996515,32cyclictest0-21swapper/723:47:207
2654599657,39cyclictest0-21swapper/520:44:565
2654499656,38cyclictest3730-21watchprog-ntop20:15:324
26544996547,17cyclictest3730-21watchprog-ntop23:50:324
47312640,0sleep10-21swapper/119:31:281
26549996415,15cyclictest0-21swapper/722:46:577
26544996445,18cyclictest26209-21chk_conn_tcp19:12:514
2654499644,36cyclictest0-21swapper/421:15:554
2653399648,27cyclictest0-21swapper/021:38:230
23472640,1sleep62344-21chk_conn_tcp22:46:196
78322630,3sleep62654699cyclictest21:09:086
26545996324,34cyclictest0-21swapper/520:37:275
2654499635,19cyclictest0-21swapper/420:55:054
26544996343,19cyclictest52-21ksoftirqd/422:06:534
26544996327,20cyclictest3730-21watchprog-ntop00:24:444
26544996325,19cyclictest16165-21chk_conn_tcp20:40:074
26544996321,36cyclictest0-21swapper/420:00:354
26546996216,45cyclictest0-21swapper/600:09:466
2654499625,56cyclictest0-21swapper/420:51:524
2653399620,1cyclictest0-21swapper/021:31:270
256982620,1sleep30-21swapper/323:27:193
57842610,2sleep618173-21QueuingRrdStrat21:41:056
313362610,2sleep52654599cyclictest19:43:175
26549996122,18cyclictest0-21swapper/720:03:477
26546996155,5cyclictest11777-21chk_conn_tcp21:54:356
26544996125,1cyclictest0-21swapper/420:27:184
26544996123,1cyclictest0-21swapper/419:00:334
2654499610,27cyclictest0-21swapper/423:14:444
26546996026,33cyclictest21538-21chk_conn_tcp19:10:426
26546996013,46cyclictest3730-21watchprog-ntop23:54:176
2654499600,35cyclictest0-21swapper/419:24:044
26536996040,0cyclictest3730-21watchprog-ntop19:23:002
26536996021,20cyclictest9788-21pool-13-thread-21:31:592
128872600,1sleep65787-21chk_conn_tcp20:03:306
125892600,0sleep30-21swapper/322:39:113
2654699590,58cyclictest0-21swapper/619:58:276
26536995939,5cyclictest27157-21openvpn23:55:532
226932590,1sleep322637-21chk_conn_tcp23:15:093
213862590,0sleep60-21swapper/620:49:376
40142580,1sleep64010-21chk_conn_tcp22:02:456
323622580,0sleep00-21swapper/021:28:170
278652580,0sleep60-21swapper/622:10:386
26549995828,1cyclictest0-21swapper/700:15:077
26549995820,31cyclictest0-21swapper/719:25:437
26546995856,1cyclictest0-21swapper/620:31:026
26544995822,14cyclictest0-21swapper/419:38:404
26544995820,18cyclictest3730-21watchprog-ntop22:26:394
2653699584,25cyclictest0-21swapper/223:01:552
219032580,0sleep30-21swapper/322:09:153
142652580,1sleep514259-21chk_conn_tcp20:04:255
119292580,0sleep30-21swapper/320:30:433
298772570,0sleep10-21swapper/122:33:561
26544995724,32cyclictest0-21swapper/400:02:504
26544995718,28cyclictest0-21swapper/423:08:204
26544995718,14cyclictest3730-21watchprog-ntop23:47:524
26544995717,21cyclictest3730-21watchprog-ntop22:49:384
26544995713,29cyclictest0-21swapper/423:42:314
2653399571,27cyclictest0-21swapper/022:20:140
81792560,0sleep20-21swapper/223:21:252
2654699564,20cyclictest25787-21chk_conn_tcp21:04:106
26545995610,29cyclictest0-21swapper/521:04:105
2654499562,17cyclictest0-21swapper/419:17:074
26544995620,16cyclictest0-21swapper/420:06:594
26544995616,39cyclictest0-21swapper/422:18:384
26536995628,21cyclictest12-21rcu_sched20:17:092
26536995617,19cyclictest0-21swapper/223:27:022
2653399560,48cyclictest0-21swapper/021:58:200
247512560,1sleep324745-21chk_conn_tcp19:26:093
26544995520,21cyclictest0-21swapper/419:46:094
26544995512,41cyclictest0-21swapper/422:39:284
26533995518,19cyclictest0-21swapper/020:49:440
26533995514,29cyclictest0-21swapper/019:03:460
147182550,0sleep40-21swapper/400:20:084
26549995441,0cyclictest0-21swapper/722:59:157
26533995452,1cyclictest188850irq/25-eth000:10:510
114732540,1sleep32654199cyclictest19:20:083
2654699539,34cyclictest3730-21watchprog-ntop20:19:496
26546995319,4cyclictest0-21swapper/622:56:026
26546995316,36cyclictest0-21swapper/622:51:146
26544995332,15cyclictest3730-21watchprog-ntop23:21:414
2653399530,18cyclictest0-21swapper/000:04:580
237522530,0sleep70-21swapper/723:37:407
2654599527,29cyclictest0-21swapper/522:55:305
26545995235,0cyclictest0-21swapper/523:26:305
26544995230,1cyclictest0-21swapper/423:09:564
26544995227,16cyclictest0-21swapper/419:58:594
26544995215,36cyclictest0-21swapper/422:30:564
2654999510,20cyclictest0-21swapper/723:40:237
26546995149,1cyclictest6713-1kworker/u17:100:27:246
26546995135,15cyclictest4673-21chk_conn_tcp21:21:486
26546995126,24cyclictest0-21swapper/623:14:126
26544995136,14cyclictest52-21ksoftirqd/422:12:454
26544995130,1cyclictest0-21swapper/422:22:234
26544995118,19cyclictest0-21swapper/421:00:254
26544995114,36cyclictest0-21swapper/420:12:524
26544995114,35cyclictest0-21swapper/421:35:114
2654499510,20cyclictest0-21swapper/422:45:534
26546995048,1cyclictest3730-21watchprog-ntop23:56:576
26546995022,27cyclictest0-21swapper/619:26:156
26533995010,20cyclictest0-21swapper/021:20:120
118642500,0sleep20-21swapper/222:27:322
118462500,0sleep30-21swapper/321:10:013
26549994926,5cyclictest0-21swapper/723:25:577
26546994921,4cyclictest19524-21chk_conn_tcp20:48:406
2653699498,24cyclictest0-21swapper/221:03:382
119682490,0sleep30-21swapper/300:10:303
2654699488,31cyclictest0-21swapper/619:23:326
2654699484,25cyclictest0-21swapper/620:26:146
2654499486,41cyclictest18789-21chk_conn_tcp00:12:274
26549994730,13cyclictest0-21swapper/720:14:287
2654999470,23cyclictest11791-21diskmemload22:11:097
2654699477,32cyclictest0-21swapper/622:25:036
2654699476,31cyclictest0-21swapper/619:53:066
26544994732,13cyclictest0-21swapper/419:41:534
6192460,0sleep50-21swapper/500:07:195
2654699469,20cyclictest3730-21watchprog-ntop21:26:366
2654699466,22cyclictest0-21swapper/620:35:196
26544994643,2cyclictest30846-21watchprog-ntop19:29:284
26544994621,24cyclictest0-21swapper/422:40:004
2654699456,0cyclictest0-21swapper/620:56:416
26546994518,25cyclictest0-21swapper/620:18:136
26546994512,19cyclictest3730-21watchprog-ntop23:59:376
26544994543,1cyclictest24560-21watchprog-ntop00:04:584
26536994543,1cyclictest13950irq/24-ahci[00022:37:522
26534994543,1cyclictest3730-21watchprog-ntop23:05:391
26534994543,1cyclictest16495-21chk_conn_tcp21:29:491
258772450,0sleep00-21swapper/020:36:490
2654699448,1cyclictest4673-21chk_conn_tcp22:35:446
26546994430,13cyclictest3730-21watchprog-ntop22:21:506
26546994428,15cyclictest20078-21chk_conn_tcp23:46:166
26546994419,24cyclictest0-21swapper/622:15:586
26545994420,21cyclictest0-21swapper/519:15:315
26544994413,30cyclictest12137-21watchprog-ntop21:10:024
2654699433,37cyclictest0-21swapper/623:36:396
26544994311,31cyclictest15339-21watchprog-ntop20:46:324
26541994327,15cyclictest43-21ksoftirqd/300:06:343
26534994341,1cyclictest21564-21VM07:43:151
26534994328,14cyclictest3730-21watchprog-ntop23:28:381
26534994323,19cyclictest3730-21watchprog-ntop21:35:431
26544994216,25cyclictest9440-21watchprog-ntop21:31:274
26544994213,1cyclictest27441-21watchprog-ntop21:04:424
2653699429,0cyclictest13950irq/24-ahci[00021:27:402
26534994241,0cyclictest0-21swapper/123:45:121
26534994240,1cyclictest3730-21watchprog-ntop22:59:471
26534994236,5cyclictest0-21swapper/119:49:541
279522410,0sleep60-21swapper/622:33:056
2654699418,1cyclictest0-21swapper/620:06:596
26546994121,19cyclictest0-21swapper/623:00:196
26546994121,19cyclictest0-21swapper/619:45:376
26546994116,24cyclictest3730-21watchprog-ntop19:32:406
26534994115,3cyclictest0-21swapper/122:40:331
2653499411,30cyclictest0-21swapper/119:47:131
2373524118,17sleep70-21swapper/718:58:257
2654999404,17cyclictest0-21swapper/720:21:257
26546994038,1cyclictest3730-21watchprog-ntop23:16:526
2654699400,39cyclictest0-21swapper/622:41:056
2653499407,32cyclictest2729-21QueuingRrdStrat21:48:001
2653499407,32cyclictest2729-21QueuingRrdStrat21:48:001
26534994038,1cyclictest27157-21openvpn21:55:081
26534994038,1cyclictest16165-21chk_conn_tcp23:17:571
26534994022,17cyclictest0-21swapper/121:25:001
26534994014,25cyclictest0-21swapper/119:37:041
2653499401,21cyclictest0-21swapper/121:06:501
26533994038,1cyclictest188850irq/25-eth020:27:180
26549993916,5cyclictest0-21swapper/719:16:037
26549993915,9cyclictest0-21swapper/722:30:237
26545993928,1cyclictest0-21swapper/520:19:175
26541993937,1cyclictest0-21swapper/322:58:433
26534993937,1cyclictest0-21swapper/119:00:331
26534993937,1cyclictest0-21swapper/100:12:591
26534993936,2cyclictest3730-21watchprog-ntop23:33:581
26534993924,1cyclictest0-21swapper/100:15:391
26534993920,18cyclictest7118-21munin-node22:14:541
26534993920,18cyclictest0-21swapper/123:57:291
26534993918,20cyclictest23218-21Poller-Thread-220:23:011
26534993916,22cyclictest27157-21openvpn20:10:121
26534993916,22cyclictest0-21swapper/120:59:211
26533993938,0cyclictest188850irq/25-eth020:41:110
2654499381,36cyclictest3730-21watchprog-ntop20:38:314
26541993836,1cyclictest0-21swapper/319:06:583
26534993831,6cyclictest0-21swapper/100:08:101
26534993819,18cyclictest2448-21openvpn21:00:571
26534993816,20cyclictest0-21swapper/122:22:231
2653499380,17cyclictest4673-21chk_conn_tcp22:50:101
26533993836,1cyclictest188850irq/25-eth023:12:360
26533993836,1cyclictest188850irq/25-eth023:05:070
26533993836,1cyclictest188850irq/25-eth020:19:490
26544993735,1cyclictest50-21rcuc/400:15:074
26541993735,1cyclictest0-21swapper/322:41:373
26541993731,0cyclictest0-21swapper/322:49:053
26541993724,12cyclictest4673-21chk_conn_tcp21:52:273
26541993719,17cyclictest27157-21openvpn23:59:053
26534993723,13cyclictest0-21swapper/119:19:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional