You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-06 - 11:39
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot1.osadl.org (updated Sun Jul 06, 2025 00:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2997221660,2sleep33251299cyclictest21:37:043
2583621550,0sleep10-21swapper/120:06:401
2410521530,0sleep00-21swapper/020:48:130
2346221510,2sleep53251599cyclictest22:41:245
3020421480,2sleep23251199cyclictest21:37:112
331321450,3sleep23251199cyclictest19:57:032
1597821390,0sleep40-21swapper/400:11:374
353721380,1sleep63251899cyclictest22:35:006
1022221340,0sleep30-21swapper/320:55:533
3054421320,0sleep10-21swapper/120:22:501
2913221320,0sleep50-21swapper/521:36:425
2704721310,0sleep70-21swapper/700:14:457
1459021280,0sleep50-21swapper/519:05:315
2005121270,1sleep224187-21Collectd-Thread21:55:382
32512991260,52cyclictest0-21swapper/320:10:373
2199821260,0sleep30-21swapper/322:52:123
32511991221,51cyclictest0-21swapper/221:49:032
375021200,0sleep10-21swapper/123:50:431
325109911938,44cyclictest0-21swapper/123:20:251
1241421190,3sleep43251399cyclictest20:43:134
325129911744,35cyclictest3730-21watchprog-ntop22:37:083
3182221170,0sleep70-21swapper/721:49:027
2717621170,0sleep30-21swapper/319:10:573
2884621160,0sleep10-21swapper/121:59:051
32511991150,58cyclictest15539-21chk_conn_tcp20:58:442
325109911543,35cyclictest5787-21chk_conn_tcp19:10:441
624521140,0sleep73252499cyclictest00:17:497
325139911416,36cyclictest0-21swapper/400:06:544
2262221140,2sleep13251099cyclictest20:33:401
1227921140,0sleep70-21swapper/723:10:427
32511991130,52cyclictest4673-21chk_conn_tcp23:19:212
325109911216,1cyclictest14342-21chk_conn_tcp22:59:031
2737021120,1sleep34673-21chk_conn_tcp20:21:163
325139911152,58cyclictest0-21swapper/419:44:574
615621100,1sleep16147-21chk_conn_tcp22:24:451
325139911019,70cyclictest16495-21chk_conn_tcp21:51:434
325099911031,78cyclictest22068-21GC07:43:150
32518991096,36cyclictest0-21swapper/600:29:536
32511991099,40cyclictest0-21swapper/223:07:362
325099910925,0cyclictest0-21swapper/022:19:300
32512991085,10cyclictest3730-21watchprog-ntop19:16:383
325129910829,15cyclictest0-21swapper/323:04:243
325119910859,33cyclictest3730-21watchprog-ntop23:39:072
325109910816,1cyclictest551-21md0_raid123:09:121
32510991081,30cyclictest25-21ksoftirqd/120:18:081
670321070,1sleep76698-21chk_conn_tcp22:24:597
353821070,0sleep70-21swapper/719:43:067
325189910730,57cyclictest3730-21watchprog-ntop19:05:246
325189910727,58cyclictest3730-21watchprog-ntop23:10:486
490721050,0sleep60-21swapper/600:17:166
325099910528,29cyclictest0-21swapper/019:32:400
3073621050,0sleep10-21swapper/123:49:311
325129910425,1cyclictest16495-21chk_conn_tcp23:55:093
325109910423,15cyclictest3730-21watchprog-ntop22:16:181
32509991040,6cyclictest0-21swapper/000:18:400
375521030,1sleep75787-21chk_conn_tcp19:29:367
325139910344,25cyclictest16165-21chk_conn_tcp23:56:454
325139910314,29cyclictest3730-21watchprog-ntop23:27:224
325129910311,28cyclictest1687-21chk_conn_tcp19:28:233
3191721030,1sleep561-21ksoftirqd/523:06:245
325189910220,40cyclictest0-21swapper/619:46:016
32512991020,40cyclictest0-21swapper/321:44:093
325119910226,20cyclictest0-21swapper/220:40:352
325119910225,36cyclictest0-21swapper/223:58:212
32513991012,59cyclictest0-21swapper/421:06:134
325109910110,8cyclictest0-21swapper/119:06:281
325119910018,2cyclictest13950irq/24-ahci[00022:11:292
325099910042,22cyclictest3730-21watchprog-ntop21:57:360
1368421000,0sleep30-21swapper/322:27:043
32511999949,37cyclictest4673-21chk_conn_tcp22:08:172
32511999921,31cyclictest0-21swapper/223:47:402
32510999923,19cyclictest3730-21watchprog-ntop19:26:471
32510999911,34cyclictest4383-21kworker/u16:200:10:391
3251899973,82cyclictest27157-21openvpn22:50:306
3251299970,35cyclictest0-21swapper/319:54:023
249042970,1sleep124901-21chk_conn_tcp19:52:301
32512999634,17cyclictest0-21swapper/322:47:183
32512999634,17cyclictest0-21swapper/322:47:183
32509999655,40cyclictest0-21swapper/021:00:530
32509999655,36cyclictest3730-21watchprog-ntop20:55:320
32509999621,49cyclictest16165-21chk_conn_tcp19:55:380
32511999531,44cyclictest4673-21chk_conn_tcp20:55:002
32509999514,33cyclictest188850irq/25-eth000:22:240
32518999432,37cyclictest23666-21DataSender-Thre23:01:116
32512999414,0cyclictest0-21swapper/321:20:073
32511999476,6cyclictest34-21ksoftirqd/222:39:172
32510999463,29cyclictest0-21swapper/100:19:441
3251299931,41cyclictest27157-21openvpn21:05:413
32511999337,25cyclictest0-21swapper/221:30:482
32511999321,31cyclictest17771-21diskmemload21:14:142
32511999312,38cyclictest0-21swapper/223:12:242
32510999325,39cyclictest0-21swapper/121:11:021
32509999333,1cyclictest188850irq/25-eth000:14:550
32509999329,2cyclictest188850irq/25-eth020:29:210
32518999211,24cyclictest0-21swapper/620:11:436
32518999211,1cyclictest0-21swapper/621:45:466
32513999210,42cyclictest3730-21watchprog-ntop23:33:154
32513999210,23cyclictest19001-21grep00:03:424
32510999222,22cyclictest10953-21nping20:14:561
32510999221,9cyclictest0-21swapper/123:38:031
303932920,0sleep70-21swapper/721:37:167
32513999110,45cyclictest0-21swapper/419:12:204
32511999128,43cyclictest34-21ksoftirqd/220:24:332
3251099918,39cyclictest0-21swapper/123:44:281
3251399904,18cyclictest0-21swapper/419:48:414
32511999036,0cyclictest5787-21chk_conn_tcp20:01:322
32511999020,41cyclictest0-21swapper/220:32:022
32509999054,0cyclictest188850irq/25-eth000:09:350
3251599895,60cyclictest3730-21watchprog-ntop20:03:095
3251399896,45cyclictest29061-21nping21:59:124
32511998932,39cyclictest0-21swapper/223:54:372
32509998930,38cyclictest4673-21chk_conn_tcp21:32:240
32518998830,0cyclictest0-21swapper/622:18:586
32515998845,24cyclictest0-21swapper/500:27:455
32515998810,31cyclictest0-21swapper/521:35:045
32513998827,5cyclictest5787-21chk_conn_tcp22:06:094
32513998827,39cyclictest0-21swapper/419:28:554
3251399880,48cyclictest0-21swapper/419:24:074
32510998851,13cyclictest3730-21watchprog-ntop21:39:211
32509998821,38cyclictest0-21swapper/023:50:210
32515998713,73cyclictest7282-21pool-14-thread-20:56:045
3251399878,42cyclictest0-21swapper/420:49:074
32513998717,0cyclictest0-21swapper/423:17:454
32511998732,40cyclictest3730-21watchprog-ntop19:04:202
32510998732,32cyclictest3730-21watchprog-ntop21:24:551
32518998649,17cyclictest3730-21watchprog-ntop21:21:436
32515998621,23cyclictest3730-21watchprog-ntop23:31:395
32513998611,38cyclictest0-21swapper/421:10:304
32512998629,13cyclictest6399-21diskmemload22:24:513
32512998625,0cyclictest0-21swapper/323:43:243
32510998622,23cyclictest24125-21Collectd-Thread19:35:201
3250999869,36cyclictest0-21swapper/019:39:360
3251899859,6cyclictest27157-21openvpn22:40:216
32518998533,40cyclictest3730-21watchprog-ntop19:01:076
32512998524,45cyclictest4673-21chk_conn_tcp23:28:583
3251199853,26cyclictest0-21swapper/220:28:172
3251099858,39cyclictest0-21swapper/121:53:511
3250999856,40cyclictest7867-21nping22:14:100
32509998544,1cyclictest9-21ksoftirqd/000:27:130
32509998526,42cyclictest0-21swapper/020:53:240
32518998428,20cyclictest0-21swapper/621:17:596
3251199848,0cyclictest0-21swapper/221:26:322
32511998430,18cyclictest0-21swapper/222:44:052
32510998418,6cyclictest0-21swapper/122:43:331
32510998415,18cyclictest27157-21openvpn19:30:311
3250999849,56cyclictest0-21swapper/020:33:060
32509998429,12cyclictest0-21swapper/022:45:090
32509998429,12cyclictest0-21swapper/022:45:090
32515998312,36cyclictest0-21swapper/521:06:455
32513998348,9cyclictest0-21swapper/422:26:594
32513998312,26cyclictest16495-21chk_conn_tcp22:17:224
3250999838,41cyclictest0-21swapper/020:20:160
32509998321,42cyclictest0-21swapper/022:41:250
32518998245,17cyclictest0-21swapper/622:01:206
3251599827,18cyclictest0-21swapper/519:40:085
32515998238,29cyclictest3730-21watchprog-ntop21:28:405
32509998228,42cyclictest188850irq/25-eth022:29:080
3251399811,36cyclictest31094-21kworker/u16:023:36:274
32512998119,59cyclictest0-21swapper/319:39:043
32511998126,3cyclictest12150-21chk_conn_tcp21:53:192
32511998120,21cyclictest13950irq/24-ahci[00000:11:432
3251899805,19cyclictest0-21swapper/620:49:396
32518998032,41cyclictest68-21rcuc/621:40:576
32515998027,39cyclictest0-21swapper/523:28:265
32513998026,11cyclictest3730-21watchprog-ntop22:55:514
32511998016,54cyclictest0-21swapper/222:00:482
32510998021,21cyclictest0-21swapper/123:55:411
3251099800,43cyclictest0-21swapper/120:28:491
32509998016,22cyclictest0-21swapper/019:43:530
3250999800,55cyclictest0-21swapper/022:00:160
3251899796,14cyclictest0-21swapper/621:29:446
32518997919,40cyclictest19134-21chk_conn_tcp21:11:026
32515997922,18cyclictest29485-1kworker/u17:022:48:225
32515997922,18cyclictest29485-1kworker/u17:022:48:225
3251599791,43cyclictest0-21swapper/520:46:595
32512997941,22cyclictest0-21swapper/320:46:273
32510997921,37cyclictest16495-21chk_conn_tcp20:03:411
3251599784,39cyclictest0-21swapper/523:02:155
32513997866,0cyclictest0-21swapper/422:21:064
32511997836,19cyclictest0-21swapper/222:16:502
3251899771,75cyclictest0-21swapper/620:29:536
32518997714,27cyclictest0-21swapper/600:06:546
32513997724,35cyclictest0-21swapper/423:11:524
32513997722,36cyclictest0-21swapper/421:27:044
32511997716,31cyclictest0-21swapper/200:05:182
32510997721,11cyclictest27923-21postmaster00:07:591
32509997739,37cyclictest14342-21chk_conn_tcp20:04:450
32509997731,35cyclictest188850irq/25-eth020:16:320
32509997718,22cyclictest479-21DefaultUDPTrans20:37:540
3250999770,15cyclictest9-21ksoftirqd/019:47:370
32518997665,10cyclictest6745-21DefaultUDPTrans22:57:596
32509997636,13cyclictest0-21swapper/019:24:390
3250999760,25cyclictest188850irq/25-eth023:17:130
282942760,1sleep024125-21Collectd-Thread22:20:360
3392750,1sleep670-21ksoftirqd/622:22:346
32515997539,35cyclictest3730-21watchprog-ntop00:04:465
32515997532,37cyclictest3730-21watchprog-ntop21:17:275
32515997512,19cyclictest0-21swapper/519:46:335
3251399756,56cyclictest0-21swapper/422:12:014
3251399752,59cyclictest0-21swapper/423:09:444
32513997511,0cyclictest0-21swapper/419:03:484
32513997510,24cyclictest12-21rcu_sched21:01:254
3251299750,1cyclictest0-21swapper/321:51:113
32512997415,58cyclictest3730-21watchprog-ntop21:04:373
32511997433,35cyclictest27157-21openvpn20:15:282
3250999744,32cyclictest0-21swapper/023:45:000
32509997440,13cyclictest0-21swapper/021:09:580
210072740,1sleep514342-21chk_conn_tcp21:56:055
106372740,1sleep232-21rcuc/220:14:422
94372730,0sleep00-21swapper/021:52:230
32513997329,30cyclictest3730-21watchprog-ntop20:37:224
32512997336,9cyclictest0-21swapper/300:02:383
322312730,1sleep40-21swapper/422:00:204
24512730,1sleep52443-21chk_conn_tcp19:00:265
32518997215,1cyclictest691ktimersoftd/620:32:346
32515997218,23cyclictest4673-21chk_conn_tcp20:24:015
32515997212,21cyclictest4673-21chk_conn_tcp22:31:485
32513997227,26cyclictest3730-21watchprog-ntop22:36:044
32513997212,15cyclictest2448-21openvpn22:54:144
32510997214,24cyclictest0-21swapper/123:02:471
267572720,1sleep126750-21chk_conn_tcp00:23:441
3251599719,1cyclictest0-21swapper/522:01:525
32515997115,21cyclictest3730-21watchprog-ntop22:20:025
3251199710,38cyclictest0-21swapper/219:13:252
95282700,0sleep70-21swapper/719:17:347
3251899700,44cyclictest5787-21chk_conn_tcp22:07:136
32515997015,0cyclictest0-21swapper/520:26:095
32513997023,6cyclictest0-21swapper/420:06:214
32512997050,0cyclictest0-21swapper/320:02:053
3251099702,43cyclictest0-21swapper/122:02:241
3251099700,7cyclictest0-21swapper/121:19:031
3251599698,23cyclictest0-21swapper/520:34:425
32515996962,6cyclictest2261-21rsyslogd19:34:165
32515996929,1cyclictest17801-21postmaster20:35:465
32515996912,37cyclictest0-21swapper/521:22:155
3251399696,42cyclictest0-21swapper/421:31:524
3251199696,36cyclictest0-21swapper/221:04:052
3251199692,36cyclictest16495-21chk_conn_tcp21:18:312
3251899681,46cyclictest0-21swapper/620:09:016
32515996816,0cyclictest0-21swapper/519:36:565
32513996826,1cyclictest0-21swapper/422:32:204
3251299680,48cyclictest4673-21chk_conn_tcp21:15:503
32511996850,1cyclictest15579-1kworker/2:0H19:08:362
32511996840,0cyclictest3730-21watchprog-ntop19:19:182
32510996837,17cyclictest25-21ksoftirqd/123:15:051
32510996812,43cyclictest25-21ksoftirqd/121:02:291
32509996824,22cyclictest3730-21watchprog-ntop22:30:440
235942680,0sleep20-21swapper/223:03:562
32518996713,1cyclictest0-21swapper/623:43:246
32512996712,20cyclictest0-21swapper/322:04:013
3251299670,32cyclictest0-21swapper/320:10:053
32511996727,6cyclictest0-21swapper/219:41:452
32510996735,31cyclictest23-21rcuc/122:33:241
32510996724,33cyclictest0-21swapper/100:28:171
3250999676,32cyclictest3730-21watchprog-ntop21:16:540
32518996615,18cyclictest0-21swapper/600:24:326
32518996611,38cyclictest0-21swapper/619:17:426
32515996621,22cyclictest0-21swapper/519:23:355
32512996630,10cyclictest3730-21watchprog-ntop20:38:263
3251299660,21cyclictest0-21swapper/322:18:263
32511996627,1cyclictest34-21ksoftirqd/221:23:192
32510996617,35cyclictest0-21swapper/121:41:291
3250999667,7cyclictest0-21swapper/022:05:370
32509996626,0cyclictest0-21swapper/020:44:190
29092660,0sleep30-21swapper/319:56:483
246082660,0sleep10-21swapper/119:24:331
60392650,3sleep73252499cyclictest23:20:127
32518996522,15cyclictest0-21swapper/622:13:066
3251599653,42cyclictest27610-21chk_conn_tcp23:57:495
3251599652,22cyclictest0-21swapper/523:20:575
32515996510,12cyclictest1383-21ssh00:15:595
3251399659,1cyclictest0-21swapper/423:01:114
32513996525,1cyclictest3730-21watchprog-ntop19:56:434
32513996510,13cyclictest0-21swapper/421:35:364
32512996530,22cyclictest3730-21watchprog-ntop20:17:363
3251299651,30cyclictest0-21swapper/321:27:363
32510996519,21cyclictest0-21swapper/122:45:411
32510996519,21cyclictest0-21swapper/122:45:411
3250999658,1cyclictest188850irq/25-eth023:45:320
32509996522,41cyclictest0-21swapper/019:55:060
3251899644,25cyclictest4158-21chk_conn_tcp21:05:416
32518996416,25cyclictest0-21swapper/623:59:586
32515996433,30cyclictest0-21swapper/522:28:365
32515996424,16cyclictest0-21swapper/521:54:245
3251599642,25cyclictest3730-21watchprog-ntop23:53:015
319112640,9sleep213950irq/24-ahci[00022:33:202
84912630,0sleep70-21swapper/719:03:087
55512630,3sleep7781ktimersoftd/720:12:077
32512996322,30cyclictest0-21swapper/319:08:043
32512996316,0cyclictest0-21swapper/323:12:563
32511996321,12cyclictest13950irq/24-ahci[00023:40:432
32510996325,20cyclictest0-21swapper/122:52:061
98612620,0sleep00-21swapper/019:17:430
3251599625,16cyclictest0-21swapper/520:12:165
32515996211,7cyclictest0-21swapper/520:07:575
32513996225,1cyclictest0-21swapper/421:21:114
32513996223,22cyclictest0-21swapper/400:22:564
32512996225,36cyclictest0-21swapper/322:34:283
32511996224,1cyclictest13950irq/24-ahci[00000:21:522
153672620,1sleep216165-21chk_conn_tcp21:43:202
97412610,1sleep123-21rcuc/119:03:501
39622610,0sleep30-21swapper/323:19:283
32518996117,0cyclictest0-21swapper/620:21:206
32518996113,46cyclictest0-21swapper/621:56:006
32515996137,16cyclictest0-21swapper/522:51:345
32515996118,33cyclictest0-21swapper/500:25:055
32515996116,43cyclictest0-21swapper/523:49:165
3251399614,56cyclictest17608-21df19:35:204
32518996035,24cyclictest550-21chk_conn_tcp19:13:576
3251899603,25cyclictest0-21swapper/620:52:206
32515996019,21cyclictest0-21swapper/522:35:325
32515996018,25cyclictest29365-21chk_conn_tcp21:47:595
3251299603,39cyclictest18673-21ssh00:21:203
3251299600,16cyclictest0-21swapper/319:00:353
58922590,1sleep55893-21chk_conn_tcp20:54:435
32518995911,24cyclictest3730-21watchprog-ntop23:27:546
32513995915,36cyclictest0-21swapper/421:16:224
132332590,0sleep40-21swapper/423:22:334
32518995835,1cyclictest0-21swapper/619:37:286
3251599583,13cyclictest0-21swapper/523:38:355
3251599580,12cyclictest0-21swapper/519:52:265
3251399580,40cyclictest0-21swapper/419:52:584
274472580,1sleep35787-21chk_conn_tcp23:05:183
32515995715,24cyclictest0-21swapper/520:41:395
32513995718,38cyclictest12272-21watchprog-ntop20:29:214
32511995721,17cyclictest34-21ksoftirqd/223:26:182
196082570,3sleep63251899cyclictest23:25:006
32518995631,17cyclictest22283-21chk_conn_tcp23:46:366
32518995611,29cyclictest0-21swapper/623:16:416
32510995635,16cyclictest4673-21chk_conn_tcp21:32:561
32510995617,0cyclictest0-21swapper/122:09:211
301672560,3sleep13251099cyclictest23:17:011
3251299550,1cyclictest27157-21openvpn20:52:203
32511995533,19cyclictest27157-21openvpn19:29:272
32510995532,1cyclictest0-21swapper/123:25:461
142752550,0sleep60-21swapper/620:01:486
98312540,2sleep23251199cyclictest23:32:252
67592540,0sleep00-21swapper/021:29:240
32518995432,21cyclictest3730-21watchprog-ntop20:19:126
32518995415,23cyclictest0-21swapper/621:36:096
32512995413,40cyclictest4329-21ssh00:17:043
32511995415,36cyclictest3730-21watchprog-ntop22:55:192
32510995410,37cyclictest0-21swapper/122:13:381
3251099540,34cyclictest0-21swapper/123:31:061
273832540,0sleep30-21swapper/321:47:213
240282540,0sleep40-21swapper/420:34:234
85782530,0sleep40-21swapper/423:52:384
32512995311,23cyclictest0-21swapper/319:20:543
313602530,1sleep716165-21chk_conn_tcp21:26:187
148252530,0sleep20-21swapper/200:29:542
58672520,0sleep63251899cyclictest19:58:126
32518995224,1cyclictest0-21swapper/600:01:026
32518995219,32cyclictest0-21swapper/619:50:506
3251899521,28cyclictest0-21swapper/619:25:436
3251199528,4cyclictest0-21swapper/220:06:532
3251899510,24cyclictest0-21swapper/622:38:136
32513995116,1cyclictest0-21swapper/421:43:054
32512995115,1cyclictest0-21swapper/319:33:123
32512995111,20cyclictest7827-21postmaster20:42:433
32509995112,17cyclictest0-21swapper/019:03:160
248122510,0sleep40-21swapper/422:41:554
32515995033,1cyclictest0-21swapper/521:01:575
32515995012,22cyclictest0-21swapper/519:12:535
32513995025,1cyclictest0-21swapper/400:18:084
3251299507,3cyclictest0-21swapper/323:30:343
32524994946,1cyclictest0-21swapper/719:34:487
32513994914,33cyclictest20204-21watchprog-ntop21:45:144
73172480,0sleep53251599cyclictest23:41:575
32515994814,2cyclictest0-21swapper/521:44:415
3251299481,19cyclictest16165-21chk_conn_tcp00:12:153
3251199486,23cyclictest13950irq/24-ahci[00022:51:022
32518994713,33cyclictest32018-21postmaster21:51:116
32515994740,6cyclictest16495-21chk_conn_tcp00:09:035
32518994639,6cyclictest0-21swapper/623:54:056
32512994611,10cyclictest14106-21pool-13-thread-23:24:423
32509994644,1cyclictest188850irq/25-eth023:12:560
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional