You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-02-25 - 07:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot4.osadl.org (updated Sun Feb 25, 2024 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
323712237174,21sleep30-21swapper/319:07:423
322722229172,17sleep70-21swapper/719:06:167
323052226170,18sleep20-21swapper/219:06:442
301792222164,18sleep40-21swapper/419:05:104
301902219188,20sleep60-21swapper/619:05:196
325132218188,19sleep10-21swapper/119:08:591
301932216163,42sleep00-21swapper/019:05:210
324892211178,21sleep50-21swapper/519:08:405
5929421190,0sleep40-21swapper/419:55:394
32938993533,1cyclictest86796-21sh20:45:016
3292399330,29cyclictest95838-21http21:00:403
32923993228,3cyclictest72945-21turbostat.cron20:20:023
3294399310,30cyclictest148774-21sh22:35:017
32938993125,5cyclictest81287-21awk20:35:026
3292399310,30cyclictest190321-21turbostat.cron23:50:013
3290799310,30cyclictest0-21swapper/021:40:000
3294399300,29cyclictest3986377-21systemd-journal20:10:017
3294399300,29cyclictest0-21swapper/723:20:387
3292399300,29cyclictest0-21swapper/323:40:473
3292399300,29cyclictest0-21swapper/323:35:383
3292399300,29cyclictest0-21swapper/319:50:383
3291699300,29cyclictest143707-21apt-get22:25:402
32907993028,1cyclictest215549-21latency_hist00:35:020
3290799300,29cyclictest53503-21apt-get19:45:360
32943992928,1cyclictest0-21swapper/720:45:427
32943992927,1cyclictest170931-21turbostat.cron23:15:017
3294399290,29cyclictest168163-21idleruntime-cro23:10:017
3294399290,28cyclictest0-21swapper/722:00:377
3294399290,28cyclictest0-21swapper/721:40:417
3294399290,28cyclictest0-21swapper/719:55:387
32938992927,1cyclictest152023-21apt-get22:40:416
3293899290,28cyclictest64120-21apt-get20:05:036
3293899290,28cyclictest199141-21apt-get00:05:406
3293899290,28cyclictest0-21swapper/623:55:406
3293899290,28cyclictest0-21swapper/623:20:396
3293899290,28cyclictest0-21swapper/619:30:396
3293899290,28cyclictest0-21swapper/619:25:366
3293899290,1cyclictest0-21swapper/600:00:436
32933992927,2cyclictest129118-21cron22:00:015
32933992927,1cyclictest165860-21apt-get23:05:385
3293399290,28cyclictest0-21swapper/519:40:405
3293399290,1cyclictest61819-21apt-get20:00:415
3293399290,1cyclictest0-21swapper/523:45:405
3293399290,1cyclictest0-21swapper/500:35:025
3292399290,1cyclictest73422-21apt-get20:20:403
3292399290,1cyclictest0-21swapper/322:35:373
3291699290,28cyclictest0-21swapper/223:05:392
3291699290,28cyclictest0-21swapper/222:45:382
3291699290,28cyclictest0-21swapper/222:05:392
3291699290,28cyclictest0-21swapper/221:25:372
3291699290,28cyclictest0-21swapper/221:20:382
32912992927,1cyclictest0-21swapper/119:55:381
3291299290,28cyclictest0-21swapper/122:40:401
3291299290,28cyclictest0-21swapper/122:20:391
3291299290,28cyclictest0-21swapper/121:05:381
3291299290,28cyclictest0-21swapper/100:25:391
3291299290,1cyclictest0-21swapper/100:00:021
3290799290,28cyclictest0-21swapper/023:05:390
3293899280,0cyclictest0-21swapper/622:00:366
32933992828,0cyclictest0-21swapper/523:50:405
32933992827,1cyclictest0-21swapper/522:25:405
32933992827,1cyclictest0-21swapper/521:10:375
32933992826,1cyclictest168629-21http23:10:415
32933992826,1cyclictest123593-21apt21:50:015
3293399280,27cyclictest76186-21apt-get20:25:395
32928992827,1cyclictest0-21swapper/423:50:404
32928992826,1cyclictest124054-21apt-get21:50:414
32928992826,1cyclictest0-21swapper/421:10:384
32928992826,1cyclictest0-21swapper/419:50:374
32928992825,2cyclictest0-21swapper/423:20:394
32923992827,1cyclictest0-21swapper/300:35:403
32923992826,2cyclictest0-21swapper/300:20:023
32923992826,1cyclictest190780-21apt-get23:50:393
3292399280,0cyclictest0-21swapper/320:35:383
32916992827,1cyclictest35897-21wc19:15:022
32907992828,0cyclictest0-21swapper/020:00:410
32907992826,1cyclictest84497-21apt-get20:40:390
3290799280,0cyclictest0-21swapper/023:15:400
32943992727,0cyclictest0-21swapper/720:35:377
32943992726,1cyclictest0-21swapper/721:20:397
3293899270,0cyclictest0-21swapper/621:30:376
32933992727,0cyclictest0-21swapper/523:15:405
32933992727,0cyclictest0-21swapper/521:00:405
32933992726,1cyclictest159863-21munin-run22:55:015
32933992726,1cyclictest0-21swapper/521:05:395
3293399270,0cyclictest0-21swapper/520:20:395
32928992727,0cyclictest0-21swapper/400:35:404
32928992725,1cyclictest0-21swapper/423:10:404
32928992725,1cyclictest0-21swapper/420:30:364
32928992725,1cyclictest0-21swapper/419:45:024
32923992727,0cyclictest0-21swapper/323:05:383
32923992727,0cyclictest0-21swapper/321:45:393
32923992727,0cyclictest0-21swapper/321:40:413
32923992727,0cyclictest0-21swapper/321:35:413
3292399270,0cyclictest0-21swapper/319:30:383
32916992727,0cyclictest0-21swapper/220:40:402
32916992726,1cyclictest0-21swapper/219:55:022
32916992726,1cyclictest0-21swapper/219:20:012
32916992726,1cyclictest0-21swapper/219:20:002
32916992725,1cyclictest0-21swapper/200:00:022
3291699270,0cyclictest0-21swapper/200:15:402
32912992725,1cyclictest0-21swapper/100:35:391
3291299270,0cyclictest0-21swapper/119:40:401
32907992727,0cyclictest0-21swapper/020:25:400
32907992727,0cyclictest0-21swapper/020:20:400
32907992725,2cyclictest101633-21apt-get21:10:370
32907992725,1cyclictest129114-21sh22:00:000
32907992725,1cyclictest0-21swapper/023:50:390
3290799270,0cyclictest0-21swapper/020:15:020
32943992626,0cyclictest0-21swapper/721:55:397
32943992626,0cyclictest0-21swapper/720:50:397
32943992626,0cyclictest0-21swapper/720:00:407
32943992625,0cyclictest0-21swapper/723:55:407
32938992626,0cyclictest0-21swapper/623:25:396
32938992626,0cyclictest0-21swapper/621:55:396
32938992626,0cyclictest0-21swapper/621:20:396
32938992625,1cyclictest0-21swapper/620:10:406
32938992624,1cyclictest47958-21apt-get19:35:406
32933992626,0cyclictest0-21swapper/520:15:025
32933992626,0cyclictest0-21swapper/519:45:365
32928992625,1cyclictest0-21swapper/400:25:384
32923992626,0cyclictest0-21swapper/323:00:393
32912992624,1cyclictest0-21swapper/123:50:401
32912992623,2cyclictest0-21swapper/120:30:371
32907992626,0cyclictest0-21swapper/023:45:390
32907992626,0cyclictest0-21swapper/019:40:390
32907992624,1cyclictest0-21swapper/021:50:010
32943992524,1cyclictest0-21swapper/719:15:407
32943992524,1cyclictest0-21swapper/719:15:407
32943992524,0cyclictest0-21swapper/700:15:397
32943992523,1cyclictest45192-21apt-get19:30:407
32938992525,0cyclictest0-21swapper/619:55:386
32933992525,0cyclictest0-21swapper/521:50:415
32933992523,1cyclictest0-21swapper/500:25:385
32923992525,0cyclictest0-21swapper/323:55:403
32923992525,0cyclictest0-21swapper/320:10:023
32923992525,0cyclictest0-21swapper/319:10:413
32923992524,1cyclictest0-21swapper/322:15:423
32923992524,1cyclictest0-21swapper/320:50:023
32923992523,1cyclictest201455-21munin-run00:10:013
32923992521,3cyclictest0-21swapper/322:25:403
32916992525,0cyclictest0-21swapper/222:10:372
32916992525,0cyclictest0-21swapper/200:20:382
32907992525,0cyclictest0-21swapper/023:40:010
32907992525,0cyclictest0-21swapper/022:25:390
32907992525,0cyclictest0-21swapper/022:20:010
32907992525,0cyclictest0-21swapper/021:40:410
32907992525,0cyclictest0-21swapper/021:20:390
32907992525,0cyclictest0-21swapper/000:15:020
32907992524,0cyclictest0-21swapper/021:00:400
32943992422,1cyclictest163095-21apt-get23:00:397
32938992424,0cyclictest0-21swapper/622:15:426
32938992424,0cyclictest0-21swapper/621:10:016
32938992421,2cyclictest3986377-21systemd-journal20:05:026
32933992424,0cyclictest0-21swapper/523:00:385
32933992424,0cyclictest0-21swapper/521:25:385
32933992424,0cyclictest0-21swapper/519:10:405
32933992424,0cyclictest0-21swapper/500:05:405
32933992423,1cyclictest0-21swapper/523:40:475
32928992424,0cyclictest0-21swapper/420:20:394
32928992424,0cyclictest0-21swapper/419:25:364
32928992423,1cyclictest0-21swapper/400:30:394
32928992423,0cyclictest0-21swapper/422:00:374
32923992424,0cyclictest0-21swapper/322:20:403
32923992423,1cyclictest0-21swapper/322:55:403
32923992423,1cyclictest0-21swapper/321:35:013
32923992423,1cyclictest0-21swapper/319:55:383
32923992422,1cyclictest0-21swapper/322:10:003
32916992424,0cyclictest0-21swapper/222:20:392
32916992424,0cyclictest0-21swapper/221:30:392
32916992424,0cyclictest0-21swapper/220:45:422
32916992424,0cyclictest0-21swapper/219:25:362
32912992424,0cyclictest0-21swapper/121:10:381
32912992424,0cyclictest0-21swapper/119:50:381
32912992423,1cyclictest0-21swapper/123:20:381
32912992423,1cyclictest0-21swapper/121:50:401
32912992422,1cyclictest42425-21apt-get19:25:371
32912992422,1cyclictest0-21swapper/120:20:401
32907992424,0cyclictest0-21swapper/021:05:380
32907992423,1cyclictest0-21swapper/023:10:410
32907992423,0cyclictest0-21swapper/021:50:410
32907992422,1cyclictest0-21swapper/000:05:400
32943992323,0cyclictest0-21swapper/723:25:387
32943992323,0cyclictest0-21swapper/722:05:397
32943992323,0cyclictest0-21swapper/721:00:407
32943992323,0cyclictest0-21swapper/700:05:407
32943992322,1cyclictest39666-21apt-get19:20:407
32943992322,1cyclictest39666-21apt-get19:20:407
32943992321,1cyclictest115023-21dump-pmu-power21:35:007
32938992323,0cyclictest0-21swapper/620:35:386
32938992321,1cyclictest78490-21apt20:30:026
32933992322,1cyclictest1-21systemd00:00:015
32933992322,1cyclictest0-21swapper/519:15:415
32933992322,1cyclictest0-21swapper/519:15:405
32933992322,0cyclictest0-21swapper/521:40:405
32928992323,0cyclictest0-21swapper/422:40:404
32928992323,0cyclictest0-21swapper/422:30:394
32928992322,1cyclictest0-21swapper/419:10:424
32928992321,1cyclictest0-21swapper/423:45:404
32923992323,0cyclictest0-21swapper/322:30:393
32923992323,0cyclictest0-21swapper/321:55:403
32923992322,1cyclictest0-21swapper/300:25:373
32916992323,0cyclictest0-21swapper/223:00:392
32907992323,0cyclictest0-21swapper/022:55:000
32907992322,1cyclictest0-21swapper/019:10:400
32943992222,0cyclictest0-21swapper/719:35:407
32943992221,1cyclictest0-21swapper/720:30:027
32938992222,0cyclictest0-21swapper/623:30:416
32938992222,0cyclictest0-21swapper/623:15:396
32938992222,0cyclictest0-21swapper/619:50:386
32938992222,0cyclictest0-21swapper/619:15:006
32933992222,0cyclictest0-21swapper/520:40:405
32933992221,1cyclictest0-21swapper/522:30:395
32933992220,1cyclictest0-21swapper/522:20:405
32928992222,0cyclictest0-21swapper/419:45:374
32923992221,1cyclictest0-21swapper/319:35:393
32923992220,1cyclictest159851-21latency_hist22:55:013
32916992222,0cyclictest0-21swapper/223:10:402
32916992222,0cyclictest0-21swapper/222:50:382
32912992222,0cyclictest0-21swapper/123:10:411
32912992221,1cyclictest0-21swapper/122:30:401
32912992221,1cyclictest0-21swapper/100:30:381
32912992220,1cyclictest0-21swapper/119:45:371
32912992220,1cyclictest0-21swapper/119:10:421
32907992221,1cyclictest0-21swapper/023:00:390
32907992221,1cyclictest0-21swapper/021:25:380
32943992120,1cyclictest0-21swapper/722:15:427
32943992120,0cyclictest0-21swapper/722:35:387
32938992121,0cyclictest0-21swapper/621:35:406
32933992119,1cyclictest0-21swapper/522:40:405
32928992120,1cyclictest0-21swapper/423:55:394
32928992120,1cyclictest0-21swapper/423:05:384
32928992119,1cyclictest121295-21http21:45:394
32916992121,0cyclictest0-21swapper/223:25:402
32916992121,0cyclictest0-21swapper/220:40:012
32916992121,0cyclictest0-21swapper/220:25:402
32916992121,0cyclictest0-21swapper/220:15:402
32916992120,0cyclictest0-21swapper/219:40:402
32916992119,1cyclictest67606-21apt-get20:10:402
32912992121,0cyclictest0-21swapper/122:00:371
32912992119,1cyclictest188019-21apt-get23:45:391
32943992020,0cyclictest0-21swapper/721:10:037
32938992020,0cyclictest0-21swapper/622:35:386
32938992018,2cyclictest126349-21sh21:55:016
32938992018,1cyclictest104399-21apt-get21:15:396
32938992017,2cyclictest148776-21sh22:35:006
32933992018,1cyclictest0-21swapper/519:55:385
32928992017,2cyclictest210043-21basename00:25:014
32923992018,2cyclictest0-21swapper/300:20:383
32916992020,0cyclictest0-21swapper/221:35:022
32916992020,0cyclictest0-21swapper/221:20:012
32916992019,1cyclictest0-21swapper/219:35:392
32916992018,1cyclictest185263-21apt-get23:40:462
32907992020,0cyclictest0-21swapper/023:40:470
32907992020,0cyclictest0-21swapper/000:25:380
32907992020,0cyclictest0-21swapper/000:00:020
32943991919,0cyclictest0-21swapper/722:40:407
32938991919,0cyclictest0-21swapper/621:25:386
32938991919,0cyclictest0-21swapper/621:15:016
32938991918,1cyclictest154785-21apt-get22:45:396
32938991918,1cyclictest0-21swapper/619:20:416
32938991918,1cyclictest0-21swapper/619:20:406
32928991919,0cyclictest0-21swapper/420:05:374
32923991919,0cyclictest0-21swapper/321:15:373
32923991919,0cyclictest0-21swapper/320:55:393
32912991918,1cyclictest0-21swapper/123:05:391
32907991919,0cyclictest0-21swapper/022:20:390
32907991919,0cyclictest0-21swapper/019:15:410
32907991919,0cyclictest0-21swapper/019:15:410
32943991818,0cyclictest0-21swapper/721:25:387
32943991818,0cyclictest0-21swapper/720:40:397
32943991818,0cyclictest0-21swapper/719:50:027
32943991818,0cyclictest0-21swapper/719:25:377
32938991818,0cyclictest0-21swapper/620:45:406
32938991818,0cyclictest0-21swapper/619:15:406
32938991818,0cyclictest0-21swapper/619:15:406
32938991817,1cyclictest146023-21munin-run22:30:006
32928991817,1cyclictest0-21swapper/422:25:414
32923991818,0cyclictest0-21swapper/321:05:393
32923991817,1cyclictest0-21swapper/322:40:403
32923991816,2cyclictest0-21swapper/321:50:393
32916991818,0cyclictest0-21swapper/223:30:422
32916991817,1cyclictest0-21swapper/222:55:402
32916991816,1cyclictest182471-21apt-get23:35:392
32912991817,1cyclictest0-21swapper/120:05:361
32907991818,0cyclictest0-21swapper/022:30:400
32943991714,2cyclictest187604-21munin-run23:45:027
32938991716,1cyclictest182471-21apt-get23:35:396
3293899170,16cyclictest157722-21cpuspeed_turbos22:50:396
32933991715,1cyclictest0-21swapper/519:50:375
3293399170,3cyclictest87645-21cat20:45:415
32928991715,1cyclictest0-21swapper/419:20:414
32928991715,1cyclictest0-21swapper/419:20:414
3292399170,16cyclictest76811-21diskstats20:25:413
3292399170,16cyclictest130218-21df_inode22:00:393
32916991717,0cyclictest0-21swapper/220:50:392
32916991717,0cyclictest0-21swapper/200:10:422
32912991715,1cyclictest0-21swapper/100:25:021
32907991717,0cyclictest0-21swapper/022:40:400
32943991616,0cyclictest0-21swapper/723:20:017
32943991616,0cyclictest0-21swapper/700:20:387
3294399160,15cyclictest0-21swapper/722:10:397
3293899160,2cyclictest91839-21python320:50:486
32933991615,1cyclictest0-21swapper/500:35:405
32928991616,0cyclictest0-21swapper/400:10:424
32928991615,1cyclictest0-21swapper/420:45:424
32928991614,1cyclictest84021-21dump-pmu-power20:40:024
32928991614,1cyclictest0-21swapper/423:43:064
32928991614,1cyclictest0-21swapper/422:35:374
32928991614,1cyclictest0-21swapper/422:05:384
3292899160,2cyclictest138906-21perl22:15:444
3292899160,15cyclictest0-21swapper/422:50:484
3292899160,15cyclictest0-21swapper/422:22:214
32923991614,0cyclictest0-21swapper/321:22:093
32923991614,0cyclictest0-21swapper/320:30:343
3292399160,15cyclictest0-21swapper/323:10:293
3292399160,14cyclictest0-21swapper/321:25:113
32916991616,0cyclictest0-21swapper/222:15:402
32916991616,0cyclictest0-21swapper/219:30:402
32912991616,0cyclictest0-21swapper/121:45:391
32912991614,1cyclictest0-21swapper/123:29:551
32912991614,1cyclictest0-21swapper/122:25:401
3291299160,2cyclictest45817-21diskstats19:30:411
3291299160,16cyclictest0-21swapper/122:50:471
3291299160,15cyclictest0-21swapper/122:45:481
3291299160,15cyclictest0-21swapper/122:15:521
3291299160,15cyclictest0-21swapper/121:25:101
3291299160,15cyclictest0-21swapper/119:37:381
3291299160,15cyclictest0-21swapper/100:10:581
3291299160,15cyclictest0-21swapper/100:05:451
32907991615,1cyclictest0-21swapper/019:55:380
32907991614,1cyclictest0-21swapper/020:05:370
32943991515,0cyclictest0-21swapper/722:45:387
32943991514,1cyclictest851rcuc/722:50:387
32943991513,1cyclictest0-21swapper/700:40:017
3294399150,15cyclictest0-21swapper/723:51:507
3294399150,15cyclictest0-21swapper/722:56:107
3294399150,15cyclictest0-21swapper/719:13:207
3294399150,14cyclictest0-21swapper/723:45:507
3294399150,14cyclictest0-21swapper/723:30:497
3294399150,14cyclictest0-21swapper/722:20:357
3294399150,14cyclictest0-21swapper/721:45:247
3294399150,14cyclictest0-21swapper/700:10:057
3294399150,0cyclictest0-21swapper/720:31:117
32938991515,0cyclictest0-21swapper/623:10:016
3293899150,15cyclictest0-21swapper/600:25:356
3293899150,14cyclictest0-21swapper/623:50:586
3293899150,14cyclictest0-21swapper/622:55:326
3293899150,0cyclictest0-21swapper/622:22:426
3293399150,15cyclictest0-21swapper/520:05:385
3293399150,14cyclictest0-21swapper/523:25:175
3293399150,14cyclictest0-21swapper/522:45:265
3293399150,14cyclictest0-21swapper/522:35:125
3293399150,14cyclictest0-21swapper/519:25:345
3293399150,14cyclictest0-21swapper/500:10:345
3293399150,14cyclictest0-21swapper/500:00:135
32928991514,1cyclictest0-21swapper/423:30:224
32928991514,1cyclictest0-21swapper/422:55:254
32928991514,1cyclictest0-21swapper/421:25:444
3292899150,1cyclictest0-21swapper/420:55:484
3292899150,14cyclictest0-21swapper/423:35:304
3292899150,14cyclictest0-21swapper/423:26:124
3292899150,14cyclictest0-21swapper/422:45:094
3292899150,14cyclictest0-21swapper/422:10:264
3292899150,14cyclictest0-21swapper/421:55:024
3292899150,14cyclictest0-21swapper/421:41:254
3292899150,14cyclictest0-21swapper/421:35:224
3292899150,14cyclictest0-21swapper/421:30:044
3292899150,14cyclictest0-21swapper/421:20:404
3292899150,14cyclictest0-21swapper/421:19:504
3292899150,14cyclictest0-21swapper/421:07:084
3292899150,14cyclictest0-21swapper/420:51:164
3292899150,14cyclictest0-21swapper/420:41:314
3292899150,14cyclictest0-21swapper/419:35:364
3292899150,14cyclictest0-21swapper/400:15:374
3292899150,14cyclictest0-21swapper/400:04:014
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional