/var/log/DMIT-NOC.log

Update: 1. HKG.Pro plan (except existing one) has been upgraded to HKG.AN5.Pro The product will be available to order very soon. HKG AN5 is using DDR5 5600 9655P hardware. 2. Half of maintenance has been completed. The rest will be completed soon.

Scheduled Maintenance Location: Hong Kong Data Center Scope: All machines Maximum simultaneous impact unit: single hypervisor Maximum impact time: each VM restarts up to 2 times, 10 minutes each. Date: June 10, 2026 - June 17, 2026 Content: Data center migration and capacity expansion. Pro (except special discount) will be upgraded to AN5 platform during this migration.

LAX: AS9929 IPv6 inbound is ready for LAX.Pro and LAX.EB

To all customers: DMIT resources are in a very critical status. That's the reason we pause all sales at this time to minimize the impact on existing customers. We do apologize that there isn't any notice about the sales pause for weeks. For all locations: Our IPv4 address purchasing is under contract, we are getting more IPs very soon. We are pushing it to move fast but we cannot guarantee when it will be ready. Also once the IPv4 is transferred, we will publish it into the resource pool to release IP demand pressure and restore sales immediately. This might bring some routing inconsistency (main for inbound.) and geolocation issues (short time advertisement, and IP database update delay). We used to hold IP for more than 3-6 mo before it was put into the resource pool but this time we couldn't. For LAX, The hardware arrived last week, due to limited space, we are working on getting the new space and power. It takes a few more weeks. Even after it's ready, we won't restore sales for large scale instances. Because we need to scale up our storage cluster; the storage cluster is on the way to datacenter and it might take a little more times more once the computing node is ready. For HKG, Hardware is onsite, we are going to build it ASAP. Once the IP resource is ready, the sales will be restored. For TYO, We are only waiting for more IPs.

AS9929 states, the packet loss and jitter caused by submarine fiber cut between Shanghai-Tokyo segment. They states (not us) it has been fixed now. Let's see how it goes.

Again: AS9929 has fragile backbone, China Unicom is reconfiguring their network recently. DMIT has report this issue since 2 days ago when it actually happened. There is no useful information and no action has been done yet. China Unicom has never change, they always like this. AS58807: CMIN2 is heavily oversell, they cannot delivery full of purchased burst bandwidth. They don't have upgrade plan in any place. To improve overall experience, DMIT will have to divert some CM traffic to CU 9929 to reduce the load for CMIN2 after AS9929 back to normal. Based on test, when the AS9929 is stable, the path of AS9808-AS4837-AS9929 will be stable and fast. There is no evidence shows cross-ISP rate limit.

Management Facility Infrastructure Maintenance DMIT.io is hosted on an independent cloud infrastructure, which is currently undergoing scheduled maintenance. Estimated maintenance duration: 1 hour.

LAX.EB AS9929: We've already report fault case with China Unicom about AS9929 issue. DMIT has enough bandwidth with China Unicom. The bandwidth expansion has been completed but DMIT will not disclosure peering capacity any more in the future. DMIT still maintains a 50% bandwidth reserve. The current packet loss and fluctuations in latency are tentatively attributed to issues with China Unicom's backbone network. AS58807: CMIN2 is heavily oversold (On China Mobile US-CN backbone end). It has reach the ceiling of their capacity. However, they still sell small amounts of bandwidth to small customers. DMIT still maintains a 20% bandwidth reserve.

Some VM restarted outside of the scheduled maintenance window. This was caused by a Kernel Panic. Following the configuration corrections and kernel component updates, a final restart will still be required during the scheduled maintenance window. We sincerely apologize for any inconvenience this may cause.

The time zone has been changed.

Scheduled Maintenance Facility: Los Angeles Data Center Scope: All Virtual Machines Maximum Simultaneous Impact: One Hypervisor at a time Maximum Impact Duration: 1 hour per client VM Time: 3:00 PM – 5:00 PM UTC Dates: April 8 – April 16 Details: Resolving anomalies caused by suboptimal kernel configurations; a system reboot is required. Note: Only one Hypervisor will be processed at a time; therefore, the overall maintenance period will be extended.

LAX EB We’ve notice the jitter issue during the peak hours. DMIT is working with upstream and CUG to resolve this matter. It should be resolved sooner. TYO PRO We’ve switched major upstream of TYO Pro to Cloudflare. There is one prefixe has issue on CTGnet routing. We’ve already reported it to CTGnet.

HKG Pro: We are currently adjusting internet routing. Some ISPs have excessively long routing convergence times, leading to detours and blackouts. Thank you for your patience.

Please note: DMIT does not own those fiber and we do not have control on China Telecom’s backbone. All the fiber cuts only impact the China Telecom network. DMIT backbone is normal so far.

TYO Pro: - NCP S3 submarine fiber cut again. - Higher latency (bypass HKG.) - No ETR. LAX Pro: - CN2 backbone local fiber cut in LA Downtown. - Packet loss and congestion happening. - ETR 14:00 UTC.

LAX-SHA CN2 GIA partial circuits failure.

LAX.EB: Observed China Unicom AS10099-9929 latency and packet loss increases. We've reached vendors about this matter. DMIT both circuits with China Unicom are both in good conditions. Based on the following observation, we thought that it could be cable fault in Unicom's network. It leads to China Unicom AS10099-9929 loss capacity, which leads to U.S. -> China direction congested. • LAX.Pro has not been impacted; • LAX. Pro shares inbound with LAX.EB ( China Unicom -> DMIT directions. ) • All China Unicom customers in Los Angeles are impacted.

Resolved. CU side fiber optic cable not seated securely. DMIT is currently upgrading the connection capacity with China Unicom. It takes a while to finish the upgrade. That's the reason why redundant interface didn't help on this fault.

We have lost one of our redundant connections to China Unicom's Los Angeles router. Due to overload, we have temporarily removed the route from AS9929 to AS4134 on Eyeball routing. This will be restored once China Unicom completes repairs.

DMIT NOC and the management team have gained a thorough understanding of the current network situation. For security reasons, DMIT will temporarily refrain from sharing further details here. DMIT is currently actively addressing the situation.