You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 17:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8s.osadl.org (updated Thu Nov 20, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30944379912340,1230cyclictest32647572sleep309:19:563
30944379911800,1176cyclictest34433162sleep311:09:563
3094437992320,1cyclictest0-21swapper/311:35:003
3094437992320,1cyclictest0-21swapper/310:00:003
3094437992118,1cyclictest0-21swapper/312:07:363
3094437992118,1cyclictest0-21swapper/310:40:003
3094437992018,1cyclictest0-21swapper/312:10:123
3094437992017,1cyclictest0-21swapper/309:10:003
3094437991916,1cyclictest0-21swapper/308:25:003
3094437991916,1cyclictest0-21swapper/307:20:143
3094437991817,0cyclictest0-21swapper/310:25:003
3094437991817,0cyclictest0-21swapper/309:53:213
3094437991817,0cyclictest0-21swapper/308:55:003
3094437991816,1cyclictest0-21swapper/312:40:003
3094437991816,0cyclictest0-21swapper/308:15:003
3094437991815,1cyclictest0-21swapper/311:18:373
3094437991815,1cyclictest0-21swapper/310:54:533
190898180,4rtkit-daemon3359395-21ssh10:15:200
3094437991715,1cyclictest0-21swapper/312:20:123
3094437991715,1cyclictest0-21swapper/308:35:143
3094437991715,0cyclictest0-21swapper/311:45:163
3094437991715,0cyclictest0-21swapper/311:45:003
3094437991714,1cyclictest0-21swapper/310:05:213
3094437991614,1cyclictest0-21swapper/312:00:213
3094437991614,1cyclictest0-21swapper/308:45:193
3094437991613,1cyclictest0-21swapper/311:10:383
3094437991613,1cyclictest0-21swapper/310:55:233
3094437991613,1cyclictest0-21swapper/310:10:223
3094437991613,1cyclictest0-21swapper/309:49:303
3094437991613,1cyclictest0-21swapper/307:35:143
35734302151,1sleep30-21swapper/312:30:163
309443799152,9cyclictest3211261-21chrt08:40:013
3094437991513,1cyclictest0-21swapper/312:30:013
3094437991513,1cyclictest0-21swapper/310:15:593
3094437991513,1cyclictest0-21swapper/310:01:373
3094437991513,1cyclictest0-21swapper/309:40:193
3094437991513,1cyclictest0-21swapper/309:21:303
3094437991513,1cyclictest0-21swapper/307:30:503
3094437991512,1cyclictest0-21swapper/311:55:163
3094437991512,1cyclictest0-21swapper/310:40:363
3094437991512,1cyclictest0-21swapper/310:30:133
3094437991512,1cyclictest0-21swapper/308:35:013
3094437991512,1cyclictest0-21swapper/308:00:173
309443799151,11cyclictest3141135-21taskset07:44:563
309443799151,10cyclictest3151226-21taskset07:50:183
3094437991413,0cyclictest0-21swapper/307:15:023
3094437991412,1cyclictest0-21swapper/311:24:043
3094437991412,1cyclictest0-21swapper/311:00:163
3094437991412,1cyclictest0-21swapper/309:28:523
3094437991412,1cyclictest0-21swapper/309:00:153
3094437991412,1cyclictest0-21swapper/309:00:003
3094437991412,1cyclictest0-21swapper/308:25:153
3094437991412,1cyclictest0-21swapper/307:30:013
3094437991411,1cyclictest0-21swapper/311:26:193
3094437991411,1cyclictest0-21swapper/307:49:073
309443799141,10cyclictest3370599-21taskset10:25:123
309443799141,10cyclictest3184316-21chrt08:15:213
309443799141,10cyclictest3100426-21taskset07:11:563
3094437991410,1cyclictest0-21swapper/307:55:113
33863082130,4sleep03386314-21ssh10:34:430
309443799134,6cyclictest32489802sleep309:10:043
309443799131,9cyclictest3508582-21taskset11:50:043
309443799131,9cyclictest3487279-21chrt11:35:173
309443799131,9cyclictest3410994-21turbostat10:50:003
309443799131,10cyclictest3295759-21chrt09:36:483
3094437991310,1cyclictest0-21swapper/309:30:123
309443799130,10cyclictest3168383-21taskset08:05:163
190898130,8rtkit-daemon3328000-21ssh09:56:120
34589092121,7sleep23458936-21apt-get11:18:372
309443799120,9cyclictest3554704-21taskset12:15:323
30883472121,5sleep03088380-21apt-get07:05:110
190898120,3rtkit-daemon3100825-21munin-node10:04:191
35803432110,8sleep13580351-21ssh12:34:061
34841962111,6sleep13484233-21apt-get11:35:111
34822492115,4sleep13417698-21kworker/u16:2+events_unbound11:31:141
33882282113,5sleep23389884-21sed10:35:172
31730602112,5sleep23173093-21apt-get08:10:122
309443799119,1cyclictest0-21swapper/307:10:013
190898110,7rtkit-daemon3266095-21irqcore09:20:151
190898110,3rtkit-daemon3363106-21sed10:20:160
35078392101,6sleep03507845-21ssh11:48:530
34275062101,5sleep23427530-21apt-get11:00:142
33867632101,5sleep13386796-21apt-get10:35:121
33609942100,7sleep13361002-21ssh10:18:011
32813512105,3sleep13281501-21sh09:30:131
32477882101,6sleep23247865-21ntp_states09:05:172
31146822101,6sleep13114782-21ntpq07:20:191
190898100,6rtkit-daemon438-21wpa_supplicant10:57:340
190898100,6rtkit-daemon3554797-21ssh12:15:440
190898100,5rtkit-daemon3410260-21ssh10:48:220
190898100,3rtkit-daemon674-21lldpd07:46:452
190898100,3rtkit-daemon3100825-21munin-node09:14:111
3582027291,5sleep03582755-21ssh12:35:180
3564271291,5sleep03564279-21ssh12:24:040
3541991291,4sleep03543170-21grep12:10:160
3524758291,4sleep13524795-21apt-get12:00:121
3485681290,6sleep23487277-21cut11:35:162
3483397291,5sleep03483405-21ssh11:33:540
3351940290,3sleep23351942-21ssh10:10:372
3346948290,6sleep03347869-21tr10:10:190
3305253290,6sleep23305332-21cat09:45:002
3263337290,1sleep13263345-21ssh09:16:341
3219005290,6sleep03220595-21kthreadcore08:45:160
3159087291,5sleep23159146-21ntp_states07:55:182
3088383290,1sleep10-21swapper/107:05:121
19089890,5rtkit-daemon667-21snmpd09:07:541
19089890,5rtkit-daemon3436838-21cut11:05:140
19089890,5rtkit-daemon3305886-21fschecks_time09:45:170
19089890,4rtkit-daemon3580875-21cat12:35:010
19089890,4rtkit-daemon3466493-21ssh11:22:201
19089890,4rtkit-daemon3402026-21ssh10:42:550
19089890,4rtkit-daemon3344251-21ssh10:06:180
19089890,4rtkit-daemon3331584-21cat10:00:160
19089890,4rtkit-daemon3287734-21ssh09:31:500
19089890,4rtkit-daemon3166848-21gltestperf08:05:142
19089890,4rtkit-daemon0-21swapper/010:54:030
19089890,2rtkit-daemon1297861-21kworker/0:1+kacpid07:33:240
3551769280,6sleep13553868-21grep12:15:181
3543171280,5sleep13545269-21cut12:10:171
3538634281,5sleep23538641-21diskmemload12:05:312
3515224280,1sleep00-21swapper/011:52:210
3452201280,2sleep13453051-21sed11:15:161
3418982280,5sleep13419140-21latency_hist10:55:001
3410112281,4sleep13410114-21ssh10:48:041
3396158281,4sleep13397301-21cut10:40:171
3327980280,2sleep13327984-21ssh09:56:091
3312168280,1sleep20-21swapper/209:47:252
3302534281,4sleep03302716-21needreboot09:40:230
3273240281,3sleep03273273-21apt-get09:25:110
3222833281,4sleep13222944-21grep08:45:221
3197310281,4sleep03197387-21ntp_states08:25:170
3193458281,4sleep1667-21snmpd08:25:141
3179337281,3sleep03179370-21apt-get08:15:150
3148220284,2sleep13149201-21kthreadcore07:50:161
3128887280,5sleep13128953-21sshd07:35:131
3128743281,4sleep03128886-21df07:35:120
3115630280,5sleep23115931-21apt-key07:25:012
3103016281,3sleep13103049-21apt-get07:15:141
19089880,5rtkit-daemon674-21lldpd07:25:560
19089880,5rtkit-daemon473-21gmain10:37:050
19089880,4rtkit-daemon81364-21packagekitd11:43:491
19089880,4rtkit-daemon667-21snmpd08:02:282
19089880,4rtkit-daemon438-21wpa_supplicant09:25:511
19089880,4rtkit-daemon3506600-21chrt11:45:572
19089880,4rtkit-daemon3289575-21diskstats09:35:110
19089880,4rtkit-daemon319-21systemd-udevd07:53:382
19089880,4rtkit-daemon3100825-21munin-node11:09:281
19089880,4rtkit-daemon3100825-21munin-node08:42:001
19089880,4rtkit-daemon1297861-21kworker/0:1+kacpid07:44:160
19089880,4rtkit-daemon0-21swapper/011:02:290
19089880,3rtkit-daemon3339042-21cut10:05:151
3587537270,5sleep23587541-21ssh12:37:032
3572056270,1sleep10-21swapper/112:28:331
3563284270,4sleep13563290-21ssh12:21:401
3490872272,3sleep03474334-21kworker/0:0+kacpi_notify11:37:350
3477073270,5sleep23477590-21kthreadcore11:30:192
3474495270,4sleep03474503-21ssh11:27:050
3474066271,4sleep23474074-21ssh11:26:122
3419310271,3sleep13419349-21apt-get10:55:121
3385737270,1sleep20-21swapper/210:33:222
3378329271,3sleep13378353-21munin-run10:29:591
3377473270,5sleep20-21swapper/210:27:562
3369960271,3sleep20-21swapper/210:25:002
3312126270,1sleep10-21swapper/109:47:201
3256097270,1sleep00-21swapper/009:13:250
3248554270,1sleep00-21swapper/009:07:560
3198495270,3sleep13198528-21apt-get08:30:111
3166741271,4sleep03166940-21hddtemp_smartct08:05:130
3137766270,4sleep13139940-21tr07:40:191
3122173271,3sleep10-21swapper/107:30:021
3104469271,3sleep23105682-21cut07:15:192
3100665270,4sleep03101533-21kthreadcore07:15:020
3094595271,3sleep03094628-21apt-get07:10:120
3090887270,4sleep23093052-21kthreadcore07:05:172
19089870,4rtkit-daemon81364-21packagekitd12:27:480
19089870,4rtkit-daemon671-21ntpd12:05:220
19089870,4rtkit-daemon667-21snmpd11:55:050
19089870,4rtkit-daemon473-21gmain10:25:160
19089870,4rtkit-daemon3210433-21runrttasks08:36:062
19089870,4rtkit-daemon3100825-21munin-node09:03:331
19089870,3rtkit-daemon81364-21packagekitd08:55:191
19089870,3rtkit-daemon671-21ntpd08:40:012
19089870,3rtkit-daemon667-21snmpd12:01:300
19089870,3rtkit-daemon667-21snmpd11:51:132
19089870,3rtkit-daemon667-21snmpd08:50:141
19089870,3rtkit-daemon667-21snmpd08:20:452
19089870,3rtkit-daemon438-21wpa_supplicant11:13:451
19089870,3rtkit-daemon3313279-21apt-get09:50:010
19089870,3rtkit-daemon3201177-21cut08:30:162
19089870,3rtkit-daemon3179540-21awk08:15:172
19089870,3rtkit-daemon3100825-21munin-node11:26:351
19089870,3rtkit-daemon1297861-21kworker/0:1+kacpid07:22:330
3582987260,3sleep13585050-21grep12:35:181
3579227260,1sleep20-21swapper/212:31:272
3548237260,1sleep20-21swapper/212:14:112
3523103260,1sleep10-21swapper/111:56:591
3508670261,2sleep10-21swapper/111:50:141
3505886260,2sleep13506018-21ssh11:45:251
3449569260,1sleep20-21swapper/211:10:452
3449222260,3sleep03449439-21users11:10:220
3443342262,2sleep20-21swapper/211:10:002
3427911260,4sleep13428070-21sed11:00:181
3410175260,1sleep20-21swapper/210:48:132
3397302260,3sleep23399339-21kthreadcore10:40:182
3369698260,4sleep10-21swapper/110:24:271
3356561260,4sleep23358609-21grep10:15:182
3345004260,3sleep23345012-21ssh10:08:062
3327276261,2sleep20-21swapper/209:55:222
3321219260,4sleep10-21swapper/109:54:091
3304015260,1sleep10-21swapper/109:42:081
3296186260,3sleep23296193-21sh09:37:432
3287443260,1sleep20-21swapper/209:31:142
3271895260,1sleep00-21swapper/009:22:380
3271758261,3sleep23271764-21ssh09:22:122
3257089260,3sleep23257128-21apt-get09:15:112
3255363260,1sleep20-21swapper/209:11:432
3236457260,4sleep20-21swapper/209:00:142
3216968260,1sleep00-21swapper/008:40:470
3094774260,3sleep23094817-21fschecks_count07:10:152
19089860,3rtkit-daemon671-21ntpd08:25:172
19089860,2rtkit-daemon3500136-21ssh11:44:402
3564958251,2sleep20-21swapper/212:25:092
3554819250,3sleep20-21swapper/212:15:472
3539250250,3sleep13539258-21ssh12:06:581
3492513250,2sleep00-21swapper/011:40:120
3463053250,3sleep00-21swapper/011:20:190
3419354250,3sleep23419502-21ssh10:55:132
3417448250,3sleep23360704-21kworker/u16:1+nfsiod10:51:202
3379083251,2sleep10-21swapper/110:30:151
3352057250,3sleep13352065-21ssh10:10:521
3335934250,2sleep2431ktimers/210:00:472
3289871250,2sleep10-21swapper/109:35:141
3278533250,2sleep23278653-21ssh09:25:222
3263268250,3sleep00-21swapper/009:16:260
3242188250,1sleep00-21swapper/009:01:280
3235311251,2sleep20-21swapper/208:55:282
3231579251,2sleep00-21swapper/008:55:180
3229650250,1sleep20-21swapper/208:52:112
3223850250,2sleep00-21swapper/008:50:130
3217852250,2sleep23218010-21if_err_eno108:45:152
3204775251,1sleep10-21swapper/108:35:151
3204218250,1sleep00-21swapper/008:31:180
3191399250,1sleep00-21swapper/008:21:250
3166541250,3sleep13166572-21apt08:05:111
3161726250,3sleep03163376-21kthreadcore08:00:170
3159259250,2sleep03159368-21find07:55:210
3148075251,2sleep00-21swapper/007:50:150
3147360250,1sleep10-21swapper/107:49:001
3140796250,1sleep20-21swapper/207:40:362
3123887250,2sleep10-21swapper/107:30:171
3100382250,1sleep10-21swapper/107:11:211
19089850,1rtkit-daemon0-21swapper/007:45:040
3556934240,3sleep20-21swapper/212:20:082
3532280240,2sleep23532288-21ssh12:04:292
3518556240,2sleep20-21swapper/211:55:162
3466523240,1sleep20-21swapper/211:22:262
3457155240,2sleep0151ktimers/011:15:240
3313789240,2sleep23313797-21ssh09:50:062
3191534240,3sleep10-21swapper/108:23:101
3180794240,2sleep13182555-21kthreadcore08:15:201
3166275240,3sleep10-21swapper/108:04:501
3158946241,2sleep13153968-21/usr/sbin/munin07:55:171
3133510240,2sleep23133814-21strings07:35:182
3128236240,3sleep20-21swapper/207:32:142
3118113240,2sleep23120212-21kthreadcore07:25:182
3205158230,1sleep00-21swapper/008:35:190
3178735230,1sleep00-21swapper/008:10:340
3178042230,1sleep10-21swapper/108:10:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional